{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Rental Listing Inquiries数据集是Kaggle平台上的一个分类竞赛任务，需要根据公寓的特征来预测其受欢迎程度（用户感兴趣程度分为高、中、低三类）。其中房屋的特征x共有14维，响应值y为用户对该公寓的感兴趣程度。评价标准为logloss。\n",
    "\n",
    "### 作业要求：\n",
    "#### 采用xgboost模型完成商品分类（需进行参数调优）。\n",
    "独立调用xgboost或在sklearn框架下调用均可。\n",
    "\n",
    "1. 模型训练：超参数调优\n",
    "\n",
    "a) 初步确定弱学xi器数目： 20分\n",
    "\n",
    "b) 对树的最大深度（可选）和min_children_weight进行调优（可选）：20分\n",
    "\n",
    "c) 对正则参数进行调优：20分\n",
    "\n",
    "d) 重新调整弱学xi器数目：10分\n",
    "\n",
    "e) 行列重采样参数调整：10分...\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 本次作业采用Sklearn与XBoosting 结合"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 导入必要的工具包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from xgboost import XGBClassifier\n",
    "import xgboost as xgb\n",
    "\n",
    "import pandas as pd \n",
    "import numpy as np\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "\n",
    "from sklearn.metrics import log_loss\n",
    "\n",
    "from matplotlib import pyplot\n",
    "import seaborn as sns\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据读取"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dpath = \"./data/\"\n",
    "train = pd.read_csv(dpath+\"RentListingInquries_FE_train.csv\")\n",
    "test = pd.read_csv(dpath+\"RentListingInquries_FE_test.csv\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 简单的查看数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>price</th>\n",
       "      <th>price_bathrooms</th>\n",
       "      <th>price_bedrooms</th>\n",
       "      <th>room_diff</th>\n",
       "      <th>room_num</th>\n",
       "      <th>Year</th>\n",
       "      <th>Month</th>\n",
       "      <th>Day</th>\n",
       "      <th>...</th>\n",
       "      <th>walk</th>\n",
       "      <th>walls</th>\n",
       "      <th>war</th>\n",
       "      <th>washer</th>\n",
       "      <th>water</th>\n",
       "      <th>wheelchair</th>\n",
       "      <th>wifi</th>\n",
       "      <th>windows</th>\n",
       "      <th>work</th>\n",
       "      <th>interest_level</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.5</td>\n",
       "      <td>3</td>\n",
       "      <td>3000</td>\n",
       "      <td>1200.0</td>\n",
       "      <td>750.000000</td>\n",
       "      <td>-1.5</td>\n",
       "      <td>4.5</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>24</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>5465</td>\n",
       "      <td>2732.5</td>\n",
       "      <td>1821.666667</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>12</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2850</td>\n",
       "      <td>1425.0</td>\n",
       "      <td>1425.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>17</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3275</td>\n",
       "      <td>1637.5</td>\n",
       "      <td>1637.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>18</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>3350</td>\n",
       "      <td>1675.0</td>\n",
       "      <td>670.000000</td>\n",
       "      <td>-3.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>28</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 228 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   bathrooms  bedrooms  price  price_bathrooms  price_bedrooms  room_diff  \\\n",
       "0        1.5         3   3000           1200.0      750.000000       -1.5   \n",
       "1        1.0         2   5465           2732.5     1821.666667       -1.0   \n",
       "2        1.0         1   2850           1425.0     1425.000000        0.0   \n",
       "3        1.0         1   3275           1637.5     1637.500000        0.0   \n",
       "4        1.0         4   3350           1675.0      670.000000       -3.0   \n",
       "\n",
       "   room_num  Year  Month  Day       ...        walk  walls  war  washer  \\\n",
       "0       4.5  2016      6   24       ...           0      0    0       0   \n",
       "1       3.0  2016      6   12       ...           0      0    0       0   \n",
       "2       2.0  2016      4   17       ...           0      0    0       0   \n",
       "3       2.0  2016      4   18       ...           0      0    0       0   \n",
       "4       5.0  2016      4   28       ...           0      0    1       0   \n",
       "\n",
       "   water  wheelchair  wifi  windows  work  interest_level  \n",
       "0      0           0     0        0     0               1  \n",
       "1      0           0     0        0     0               2  \n",
       "2      0           0     0        0     0               0  \n",
       "3      0           0     0        0     0               2  \n",
       "4      0           0     0        0     0               2  \n",
       "\n",
       "[5 rows x 228 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['bathrooms', 'bedrooms', 'price', 'price_bathrooms', 'price_bedrooms',\n",
       "       'room_diff', 'room_num', 'Year', 'Month', 'Day',\n",
       "       ...\n",
       "       'walk', 'walls', 'war', 'washer', 'water', 'wheelchair', 'wifi',\n",
       "       'windows', 'work', 'interest_level'],\n",
       "      dtype='object', length=228)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['bathrooms', 'bedrooms', 'price', 'price_bathrooms', 'price_bedrooms',\n",
       "       'room_diff', 'room_num', 'Year', 'Month', 'Day',\n",
       "       ...\n",
       "       'virtual', 'walk', 'walls', 'war', 'washer', 'water', 'wheelchair',\n",
       "       'wifi', 'windows', 'work'],\n",
       "      dtype='object', length=227)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "y_train = train[\"interest_level\"]\n",
    "X_train = train.drop(\"interest_level\",axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 因为已经做了数据处理，所以直接进行模型训练"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在树模型中有三个参数特别重要：树的数目、树的深度和学习率，然后我们采用以下方式来进行模型训练\n",
    "– n_estimators和learning_rate：固定n_estimators为100（数目不大，\n",
    "因为树的深度较大，每棵树比较复杂），然后调整learning_rate\n",
    "– 树的深度max_depth：从6开始，然后逐步加大\n",
    "– min_child_weight：1⁄𝑠𝑞𝑟𝑡3 rare_events，其中rare_events为稀有\n",
    "事件的数目\n",
    "– 列采样colsample_bytree／ colsample_bylevel：在[0.3, 0.5]之间进行\n",
    "网格搜索\n",
    "– 行采样subsample：固定为1\n",
    "– gamma: 固定为0.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. 直接调用xgboost内嵌的cv寻找最佳的参数n_estimators"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# prepare cross validation\n",
    "kfold = StratifiedKFold(n_splits=5, shuffle=True, random_state=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#直接调用xgboost内嵌的交叉验证（cv），可对连续的n_estimators参数进行快速交叉验证\n",
    "#而GridSearchCV只能对有限个参数进行交叉验证\n",
    "def modelfit(alg, X_train, y_train, cv_folds=None, early_stopping_rounds=10):\n",
    "    xgb_param = alg.get_xgb_params()\n",
    "    xgb_param['num_class'] = 3 #分类数目当使用 multi:softmax时使用\n",
    "    \n",
    "    #直接调用xgboost，而非sklarn的wrapper类\n",
    "    xgtrain = xgb.DMatrix(X_train, label = y_train)\n",
    "        \n",
    "    cvresult = xgb.cv(xgb_param, xgtrain, num_boost_round=alg.get_params()['n_estimators'], folds =cv_folds,\n",
    "             metrics='mlogloss', early_stopping_rounds=early_stopping_rounds)\n",
    "  \n",
    "    cvresult.to_csv('1_nestimators.csv', index_label = 'n_estimators')\n",
    "    \n",
    "    #最佳参数n_estimators\n",
    "    n_estimators = cvresult.shape[0]\n",
    "    \n",
    "    # 采用交叉验证得到的最佳参数n_estimators，训练模型\n",
    "    alg.set_params(n_estimators = n_estimators)\n",
    "    alg.fit(X_train, y_train, eval_metric='mlogloss')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#params = {\"objective\": \"multi:softprob\", \"eval_metric\":\"mlogloss\", \"num_class\": 9}\n",
    "xgb1 = XGBClassifier(\n",
    "        learning_rate =0.1,\n",
    "        n_estimators=1000,  #数值大没关系，cv会自动返回合适的n_estimators\n",
    "        max_depth=5,\n",
    "        min_child_weight=1,\n",
    "        gamma=0,\n",
    "        subsample=0.3,\n",
    "        colsample_bytree=0.8,\n",
    "        colsample_bylevel=0.7,\n",
    "        objective= 'multi:softprob',\n",
    "        seed=3)\n",
    "\n",
    "modelfit(xgb1, X_train, y_train, cv_folds = kfold)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEXCAYAAABCjVgAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xm8XHV9//HXZ2buvuSG5IaEJBCW\nsETLZiAgglFEWawoVQQFC7WlbnWvYmvVqliXlqo/F6QU0IpQWhApIIqKBmQNGJYkLElIwiXbTXJv\ncvdtPr8/vmduJjczd8ude+69834+Hiczc853zvmcOTfznrObuyMiIgKQiLsAERGZOBQKIiLST6Eg\nIiL9FAoiItJPoSAiIv0UCiIi0k+hIJLFzP7BzK6Luw6RuCgUJhkzqzaz9Wb2nqx+NWa20czemdVv\nsZndZWZNZtZsZqvM7Cozmx4Nv8zM+sysNerWmdkHC1z7UjNrKOQ0RiJXPe7+NXf/6wJNb72ZvakQ\n4y6E8Vpek+1zmeoUCpOMu7cCVwDfMbP6qPc3geXu/r8AZvZa4PfAH4Gj3b0OOBvoBY7LGt3D7l7t\n7tXAO4FvmtkJ4zMnMhJmloq7BikS7q5uEnbAjcDNwFJgBzAna9iDwP8b4v2XAQ8O6PcY8J6s128D\nVgLNhJA5JmvYMVG/5qjN27KGnQusAlqAV4BPA1VAB5AGWqPuoDzz9X3g7uj9jwKHD+PzOBq4D9gJ\nPA9cOJp6gC8BP43etwBw4HLgZaAJ+ABwEvB0NO/fy5rO4cDvouWxHbgJqIuG/Vc0rY5oWp8Zxme8\nHvhsNK0uIBW9fiWal+eBM3N8FqcAW4BkVr93AE9Hz08GlgO7ga3A1Xk+06VAQ55h04CfAI3ABuDz\nQCIalgT+LfoMXgI+En2OqTzjWg+8Kc+wvwHWRMv1zszfDGDAvwPbgF3RZ/TqfMs77v+vk6mLvQB1\no1xwMB3YHP3HuzyrfxXQBywd4v2XkRUK0RddM3Bk9PpIoA04CygBPhP95yyNXq8B/iF6/cboP+BR\n0Xs3A6dn1Xli9Dzvl0xWHTdGXwAnR1+CNwG3DPGeKsKX9uXRe06MPpdXjbQecofCNUA58GagE7gD\nmAXMjb6UXh+1PyL6vMqAemAZ8O2sce/15TfYZ5zVfgUwH6gAjorm86Cs+nIGJrAWOCvr9f8AV0bP\nHwYujZ5XA6fkGUfe5UUIhF8ANVEdLwDvj4Z9gPClPC/6vH/DKEIh+rvaHi3PMuD/AcuiYW8BngDq\nCAFxDNEPo3zLW93wOm0+mqTcvYnwC7MSuD1r0HTCZsEtmR5m9s1ov0KbmX0+q+0pUf9WwlrCfwEv\nRsPeDdzt7ve5ew/wr4QvptcSfolWA1939253/x1wF3Bx9N4eYJGZ1bp7k7s/OcLZu93dH3P3XkIo\nHD9E+7cC6939BnfvjaZ3G2GT2FjU8xV373T3XxO+xG92923u/grwAHACgLuviT6vLndvBK4GXj/I\neAf7jDO+6+4vu3sHIezLonkpcff17r42z7hvJloeZlZD+PV8c9bncYSZzXT3Vnd/ZCQfhpklo9o/\n5+4t7r6esGZwadTkQuA77t4Q/Z1+fSTjz/Je4Hp3f9Ldu4DPAaea2YJoHmoIa4jm7qvdfXPW/O3P\n8i5qCoVJyswuIfxC+w3wjaxBTYTNFHMyPdz9Mx72K/yc8Es64xF3r/OwT2E28Crga9GwgwibBTLj\nSBN+pc6Nhr0c9cvYEA0D+AvCl9AGM/uDmZ06wtnbkvW8nRBAgzkEWBIFXLOZNRO+UGaPUT1bs553\n5HhdDWBms8zsFjN7xcx2Az8FZg4y3sE+44yXs4avAT5OWJvZFk3roDzj/hlwgZmVARcAT7p7Zlrv\nJ6ylPGdmj5vZWwepMZeZhDXEDVn9spf/Qdl1D3g+EgM/n1bCprm50Q+R7xE2NW41s2vNrDZqur/L\nu6gpFCYhM5tF2J76N8DfAhea2RkA7t5G2A5/wUjG6e5bCb+u/zzqtYnwZZuZphE2Y7wSDZtvZtl/\nPwdHw3D3x939fMImljuAWzOTGUlNI/Ay8Ico4DJdtbt/cJzr+ZdonMe6ey1wCWHTRsbA6Q32Ged8\nj7v/zN1fF73P2fsHQXa7VYQv1HOA9xBCIjPsRXe/mPB5fAP4XzOrGv5ssp3wa/yQrH79y5+w+WZe\n1rD5Ixh3toGfTxUwgz1/Z99199cQfswcCfx91D/f8pZhUChMTt8D7nD3+6NV5s8A/xH9KiR6/Vdm\ndmUUIJjZPODQfCM0sxmEnZEro163AueZ2ZlmVgJ8irCz8yFC6LQBnzGzEjNbSgiTW8ys1Mzea2bT\nok0iuwmbPSD8wp5hZtPG6HPIuAs40swujeopMbOTzOyYca6nhrATudnM5hJ9SWXZChyW9Xqwz3gf\nZnaUmb0xWs6dhLWUvlxtIz8DPgqcQdinkBnPJWZWH62ZNEe9847HzMqzO8Ka6K3AVdHh0IcAnySs\nGWXm62NmNtfM6gg7x4dSMmA6qaj+y83s+GievwY86u7ro+W7JPrc2qLPo2+I5S3DEfdODXUj64C3\nE35B1Q3o/1vgqqzXS4B7CP/pm4FngauAGdHwywj/WTJH3mwjbHOelTWOdxB2GO4C/kC04zYa9qqo\n366ozTui/qXAvYTNWLuBx4HXZb3vesImgGbyH3301azXSxli53TU7ijCEUuN0fh/R9gXMaJ6yL2j\nOZXVvoGsnfiEL8LPZ30mT0Sf5wrCl3xDVtvzgY3RtD49jM94PXvvmD6WsO+nhbAz/q5cn2FW+4MJ\nX+B3D+j/02h5txJ+BLw9z/uXRvM/sDuCsO/qp9Hn/TLwBfYcfZQirMnuIBx99AnCmoXlmc76HNP4\najTsA4Sd5pn5nRf1P5NwxFEre470qh5qeasburPoAxYRKQgzOwe4xt0PGbKxxE6bj0RkTJlZhZmd\na2apaDPaFwkHOcgkoDUFmRTM7HTgl7mGeTh6SiYIM6skbAo7mrDf427gY+6+O9bCZFgUCiIi0k+b\nj0REpN+ku8jWzJkzfcGCBXGXISIyqTzxxBPb3b1+qHYFCwUzm0+4PspswmFx17r7dwa0MeA7hLMP\n24HLfIhT0hcsWMDy5csLU7SIyBRlZhuGblXYNYVe4FPu/mR07ZUnzOw+D2daZpwDLIy6JcAPo0cR\nEYlBwfYpuPvmzK9+d28BVrP3NV0gnMzzEw8eAerMbA4iIhKLcdnRHF3V8ATC5RGyzWXvi2U1sG9w\niIjIOCl4KJhZNeFCax/PcZyy5XjLPsfImtkVZrbczJY3NjYWokwREaHAoRBdrOo24CZ3vz1Hkwb2\nvoLiPMJ1ffbi7te6+2J3X1xfP+TOcxERGaWChUJ0ZNF/Aqvd/eo8ze4E3mfBKcAu33OjDBERGWeF\nPProNMKdmJ4xsxVRv38gXLkRd7+GcBXPcwm3IGwn3E5RRERiUrBQcPcHyb3PILuNAx8uVA3Ztu7c\nxep16zlp0ZFUVVaMxyRFRCadornMxdPffRdL7zqDbRtWx12KiMiEVTShcOAZ7wegs21XzJWIiExc\nRRMKpZXhnt497bp6r4hIPsUTClXhNrwKBRGR/IomFCqiUOjtbIm5EhGRiatoQqG8OoRCWqEgIpJX\n0YRCZXXYp+BdrTFXIiIycRVNKJSWV9PnBt1aUxARyadoQsESCdqtHOvWmoKISD5FEwoAHVSS6GmL\nuwwRkQmrqEKhM1FBSqEgIpJXUYVCV6KSVJ9CQUQkn6IKhe5kJSV9HXGXISIyYRVVKPQmKynra4+7\nDBGRCau4QqGkinJXKIiI5FNUoZAuqaLCtflIRCSfIguFaiq9g3BvHxERGaioQoGyasqth67u7rgr\nERGZkAoWCmZ2vZltM7Nn8wyfZmb/Z2ZPmdlKMyv4/ZmtrAaA9pbmQk9KRGRSKuSawo3A2YMM/zCw\nyt2PA5YC/2ZmpQWsh0R5CIWOVt1TQUQkl4KFgrsvA3YO1gSoMTMDqqO2vYWqByBVXg1Ah27JKSKS\nUyrGaX8PuBPYBNQA73b3dCEnmKoIl8/ublcoiIjkEueO5rcAK4CDgOOB75lZba6GZnaFmS03s+WN\njY2jnmBJhe7TLCIymDhD4XLgdg/WAC8BR+dq6O7Xuvtid19cX18/6glm7r52+8PPjXocIiJTWZyh\nsBE4E8DMDgSOAtYVcoKVNdMBOHthVSEnIyIyaRVsn4KZ3Uw4qmimmTUAXwRKANz9GuArwI1m9gxg\nwGfdfXuh6gGomjYDgHSHDkkVEcmlYKHg7hcPMXwT8OZCTT+XiprppN2gQzuaRURyKaozmi2RpM0q\nsC6tKYiI5FJUoQDQZtUku1viLkNEZEIqulBoT1RT0qNDUkVEcim6UOhM1VDaqzUFEZFcii4UulM1\nVPQpFEREcim6UOgrraUq3RZ3GSIiE1LRhUK6rJZqb9ONdkREcii6UPDyOqqtg84u3WhHRGSgoguF\nREW4/lHLrh0xVyIiMvEUXSgkK+sAaFcoiIjso+hCoaQqXBSvffdg9/8RESlORRcKZTXhonhdrVpT\nEBEZqOhCoaLmAAC625pirkREZOIpulCoqg2h0Kf7NIuI7KPoQqG6Lmw+6tM9FURE9lF0oVBSUUuP\nJ7EObT4SERmo6EIBM3ZbDYlOhYKIyEDFFwpAa3IaJV0KBRGRgQoWCmZ2vZltM7NnB2mz1MxWmNlK\nM/tDoWoZqCNVS0WP9imIiAxUyDWFG4Gz8w00szrgB8Db3P1VwLsKWMteukqnU9mno49ERAYqWCi4\n+zJgsNOG3wPc7u4bo/bbClXLQL1l06lN6+5rIiIDxblP4Uhgupn93syeMLP35WtoZleY2XIzW97Y\n2LjfE/aKA6ijha6e3v0el4jIVBJnKKSA1wDnAW8B/snMjszV0N2vdffF7r64vr5+vydsVTNIWZpd\nTbrUhYhItjhDoQG4193b3H07sAw4bjwmnKqeCcDunVvHY3IiIpNGnKHwC+B0M0uZWSWwBFg9HhMu\nrQmh0N48brsxREQmhVShRmxmNwNLgZlm1gB8ESgBcPdr3H21md0LPA2kgevcPe/hq2Opsm4WAF27\n9n//hIjIVFKwUHD3i4fR5lvAtwpVQz7VBxwIQHfr9vGetIjIhFaUZzTXTA+hkFYoiIjspShDobSq\njl5PQLvuviYikq0oQ6H/oni6/pGIyF6KMxSAVsopbV4XdxkiIhNK0YZCW/lBVCbTcZchIjKhFG0o\ndJfPoLZP+xRERLIVbSj0Vs7iAG+mp09rCyIiGUUbClY9iyrroqlJO5tFRDKKNhRKps0GoLmxIeZK\nREQmjqINhYrpIRRat2+KuRIRkYmjaEOhesZcADqaN8dciYjIxFG0oTCtfh4Avbu2xFyJiMjEUbSh\nUDFtFmk3vFVXShURySjaUCCZojlRS6pD91QQEcko3lAAWpLTKevULTlFRDKKOhTaS2dS1aNQEBHJ\nKOpQ6K6oZ3rfDtw97lJERCaEog4Fr57DTJrZ3dEddykiIhNCwULBzK43s21mNuh9l83sJDPrM7N3\nFqqWfJJ1cymxPhq36KxmEREo7JrCjcDZgzUwsyTwDeBXBawjr4oZ4VyFXVvWxzF5EZEJp2Ch4O7L\ngKGuTf13wG1ALMeF1sw6BID2HS/HMXkRkQkntn0KZjYXeAdwzTDaXmFmy81seWPj2J1sNn32AgB6\nmrT5SEQE4t3R/G3gs+7eN1RDd7/W3Re7++L6+voxK6C09kB6SWItuv6RiAhAKsZpLwZuMTOAmcC5\nZtbr7neMWwWJBDsTB1DapusfiYhAjKHg7odmnpvZjcBd4xoIkd0ls6js0qUuRERgGKFgZocDDe7e\nZWZLgWOBn7h78xDvuxlYCsw0swbgi0AJgLsPuR9hvHRWHMj05tVxlyEiMiEMZ03hNmCxmR0B/Cdw\nJ/Az4NzB3uTuFw+3CHe/bLhtx1pf9RxmNf2R1s4eqstL4ipDRGRCGM6O5rS79xKOFPq2u38CmFPY\nssZPcvp8Kq2LzVu0s1lEZDih0GNmFwN/CdwV9ZsyP6mrZoVdG02vrIm5EhGR+A0nFC4HTgWucveX\nzOxQ4KeFLWv81B10BACtW9fFXImISPyG3Kfg7quAjwKY2XSgxt2/XujCxkvdnMMB6Nu5IeZKRETi\nN+Sagpn93sxqzewA4CngBjO7uvCljQ+rqKONSpItutSFiMhwNh9Nc/fdwAXADe7+GuBNhS1rHJmx\ns+RAqtpfibsSEZHYDScUUmY2B7iQPTuap5TWioOo69kadxkiIrEbTih8mXBp67Xu/riZHQa8WNiy\nxldvzXzm+DZadLMdESlyQ4aCu/+Pux/r7h+MXq9z978ofGnj58HtldRaB69s0TWQRKS4DWdH8zwz\n+3l0F7WtZnabmc0bj+LGy3mnLwFgx8vPx1yJiEi8hrP56AbCpS0OAuYC/xf1mzJmzD8KgLYtOoFN\nRIrbcEKh3t1vcPfeqLsRGLubGkwAVbMXApDesTbmSkRE4jWcUNhuZpeYWTLqLgF2FLqwcVVWTVNi\nOmUtOoFNRIrbcELhrwiHo24BNgPvJFz6YkppKptHXYduyykixW04Rx9tdPe3uXu9u89y97cTTmSb\nUjprFzAnvZmO7iHvDioiMmWN9h7NnxzTKiYAm3EYs62Jl7duj7sUEZHYjDYUbEyrmAAqDww7m791\ny70xVyIiEp/RhoKPaRUTwMyDw2Gp7+j7VcyViIjEJ28omFmLme3O0bUQzlkYlJldH53w9mye4e81\ns6ej7iEzO24/5mO/Vc05GoB0zZCzJiIyZeW9n4K71+znuG8Evgf8JM/wl4DXu3uTmZ0DXAss2c9p\njl55LTuTM6ncpXMVRKR4jXbz0ZDcfRmwc5DhD7l7U/TyESD2S2c0VR7KzK4NpNNTbuuYiMiwFCwU\nRuj9wC/zDTSzK8xsuZktb2xsLFgRPdOP4FA2sXlXR8GmISIykcUeCmb2BkIofDZfG3e/1t0Xu/vi\n+vrCXWGjbPbR1FgHGzfoGkgiUpxiDQUzOxa4Djjf3WO/dMYBC/4MgOb1OfeNi4hMeXl3NGdERxsN\n3Mi+C1gOfMrd141mwmZ2MHA7cKm7vzCacYy12nmLAOjasjrmSkRE4jFkKABXA5uAnxFOWrsImA08\nD1wPLM31JjO7ORo208wagC8CJQDufg3wBWAG8AMzA+h198Wjn5X9ZzWzaUnUUNY0ITJKRGTcDScU\nznb37ENFrzWzR9z9y2b2D/ne5O4XDzZSd/9r4K+HWef4MGNH1UJm715Lb1+aVDL2XS4iIuNqON96\naTO70MwSUXdh1rApd+xm78xFHMlGXmpsibsUEZFxN5xQeC9wKbAt6i4FLjGzCuAjBawtFpXzj6PK\nuli/dlXcpYiIjLshNx9FO5L/PM/gB8e2nPjVH3EiLIOW9U/BaafGXY6IyLgack3BzOaZ2c+j6xht\nNbPbzCz2s48LpWT2ItIYbNVhqSJSfIaz+egG4E7CRfDmAv8X9ZuaSivZYdOpa34W9ym3y0REZFDD\nCYV6d7/B3Xuj7kagcKcVTwCts5dwlG2koUmXuxCR4jKcUNhuZpeYWTLqLgFiP/u4kMrmn8hc28Hz\na0d1Xp6IyKQ1nFD4K+BCYAuwGXgncHkhi4rbzKNOAWDnmsdirkREZHwNGQruvtHd3+bu9e4+y93f\nDlwwDrXFpnTu8eHJphXxFiIiMs5Ge8ruJ8e0iommvJbtZfOZsXsVfbq3gogUkdGGgo1pFRNQR/1x\n/Bkv8tzmXXGXIiIybkYbClP+53PVwtcxy5p54bln4i5FRGTc5A0FM2sxs905uhbCOQtT2vSjzwCg\nbc2UO2lbRCSvvJe5cPea8SxkorH6Y2hL1DBt2/K4SxERGTe6NnQ+iQQ7DjiBRT0reaVZJ7GJSHFQ\nKAyirHM7hyc28+Qq3XRHRIqDQmEQ9e/6dwB2rF4WcyUiIuNDoTCIxNwT6LZSyjbpzGYRKQ4FCwUz\nuz663HbOa1Bb8F0zW2NmT5vZiYWqZdRSZayyhRzT8ywbdrTFXY2ISMEVck3hRuDsQYafAyyMuiuA\nHxawllFbcOKbeLWt58GVG+IuRUSk4AoWCu6+DNg5SJPzgZ948AhQZ2ZzClXPaNUds5SUpdn8zG/j\nLkVEpODi3KcwF3g563VD1G8fZnaFmS03s+WNjY3jUly/g19Ld6Kc2VuX0d7dO77TFhEZZ3GGQq7r\nJ+W8fIa7X+vui919cX39ON/fp6Sc1jmn8Xr+xB9f3D6+0xYRGWdxhkIDMD/r9TxgU0y1DKr22POY\nn2jkmacej7sUEZGCijMU7gTeFx2FdAqwy903x1hPXqmj3gxAYu19um+ziExpea99tL/M7GZgKTDT\nzBqALwIlAO5+DXAPcC6wBmhnIt/NrW4+zTULWdy8nJWbdvPqudPirkhEpCAKFgrufvEQwx34cKGm\nP9ZKj34LJz/2Q877yYPc97nz4i5HRKQgdEbzMFW+6hxKrY/TkjnPxRMRmRIUCsM1fwndqRpe3fIg\nL25tibsaEZGCUCgMV7KEviPP4azEcu5ZsTHuakRECkKhMAIVx13ANGtn45P3kk7rKCQRmXoUCiNx\n2BvoSVWxpO0PPLxuR9zViIiMOYXCSJSUY4vO57zUo/z8Ud14R0SmHoXCCKVe8z6q6CS5+k6a2rrj\nLkdEZEwpFEbq4FPomnYoFyTu544Vr8RdjYjImFIojJQZZYsvZUniOZY9/KgueyEiU4pCYTSOu5g0\nCU5s/iUP6MqpIjKFKBRGo/Yg/IgzeXdqGT+6/7m4qxERGTMKhVFKnvR+ZrGTGRt+yZMbm+IuR0Rk\nTCgURmvhW0jPWMiHSu/mB79bE3c1IiJjQqEwWokECXeOZj0dL/yW57bsjrsiEZH9plDYHx96iHTV\nLD5Ueg9f/6X2LYjI5KdQ2B+pMhKnfIDTeIptLzzOAy82xl2RiMh+USjsr8V/hZdP458qbuOqu1fT\npwvlicgkplDYXxXTsdd9klPTT1C37VH+Z/nLcVckIjJqBQ0FMzvbzJ43szVmdmWO4Qeb2f1m9icz\ne9rMzi1kPQWz5G/x2rl8tepW/uWe1Wxr6Yy7IhGRUSlYKJhZEvg+cA6wCLjYzBYNaPZ54FZ3PwG4\nCPhBoeopqJIK7A3/yBE9L7C074984Y6VuvyFiExKhVxTOBlY4+7r3L0buAU4f0AbB2qj59OATQWs\np7COuwgOfDVfrfxvHlj5Evc8syXuikRERqyQoTAXyN7A3hD1y/Yl4BIzawDuAf4u14jM7AozW25m\nyxsbJ+gRPokkvPXfqe7aypVl/8vHbvkTm3d1xF2ViMiIFDIULEe/gdtULgZudPd5wLnAf5nZPjW5\n+7XuvtjdF9fX1xeg1DEy/2TspL/mEruXxal1fOimJ+nuTcddlYjIsBUyFBqA+Vmv57Hv5qH3A7cC\nuPvDQDkws4A1Fd6ZX8Bq5vC11LWs3NjIVXevirsiEZFhK2QoPA4sNLNDzayUsCP5zgFtNgJnApjZ\nMYRQmKDbh4apvBbe+u8clt7ATw++mx8/vIE3fOv+uKsSERmWgoWCu/cCHwF+BawmHGW00sy+bGZv\ni5p9CvgbM3sKuBm4zKfCYTtHnQ2nfIiTt93KJ+e/wIad7dz7rHY8i8jEZ5PtO3jx4sW+fPnyuMsY\nWm83XP9mfOc6Plh1Nb/bWsWPLz+ZUw+fEXdlIlKEzOwJd188VDud0VwoqVJ45w0Yxvf960y3Nt57\n3SMse2Fybx0TkalNoVBIBxwKF/2M5M41PFjxaWpK0vzl9Y9pU5KITFgKhUJbcBr8xXWUdDex/Jhb\nqS2DD/z0Ca5dtlZnPYvIhKNQGA9/9k54y9coef5Onlh0K/WVSb52z3N84r9X0NnTF3d1IiL9FArj\n5dQPw1lfIbX6Dh475hY+e9Zh3LFiE6/5yn00NLXHXZ2ICKBQGF+nfRTO+gq28ud8sOGz3HDRUXT0\n9HHGN+/nlsc2anOSiMROoTDeTvsovONHsOGPvOGh9/Hg3x7BkkNncOXtz3Dpfz7Gmm2tcVcoIkVM\noRCH4y6CS26DXQ0c9N9nc9Mb2vnK+a/i4XU7OOvqP3DV3atoauuOu0oRKUIKhbgcthSuuB9qZpO4\n6QIu7byZRz97BjOrS/mPB17i9G/ez9W/fp5dHT1xVyoiRURnNMetuw3u+iQ8fQvMPhbecQ3P+Xy+\n85sX+eWzW0gmjI++cSGXv24BteUlcVcrIpPUcM9oVihMFKvuhLs+AZ27YOln4bUfZeW2Tr79mxe5\nb9VWEgbvPmk+F510MMfOm4ZZriuTi4jkplCYjNq2w/eXQPt2mL4A3vTPsOh8nt20mx8/tJ7bnmwg\n7bBoTi0Xnzyf8449iAOqSuOuWkQmAYXCZLbmN/Drf4Jtq2D+EnjzVTD/JHZ39vCLFZu45bGNrNy0\nG4Ca8hQfO3MhZy06kENmVMVcuIhMVAqFya6vF1bcBPdfBa1b4Zg/h1P/DuafDGas3LSLXz27hese\nfIn27nBW9JEHVvPaw2dyymEzWHLoAUzXWoSIRBQKU0VXKzz0XXjgXyHdB3NfA6d8CBadD8mw4/nl\nne38etVW7n9uGw+t3U46WqQVJUn+4jVzOXZeHcfOm8YR9dWkkjrgTKQYKRSmmq5WeOpmeOQHsHMd\nJMvg9E/B8RdD3cH9zbp70zzzSjOPrNvJI+t28NCaHfRFyzhhUFma4l2L53HM7FoOn1XF4fXV1FVq\njUJkqlMoTFXpNLz4K3jkh/DSH0K/Q8+AYy+Co86BygMGNHde2tHGMw27eKqhmWcadvHkxqb+tQmA\nVMI44eA6Dq+v5rD6EBQHH1DJrNpyastTOtJJZAqYEKFgZmcD3wGSwHXu/vUcbS4EvgQ48JS7v2ew\ncRZ9KGRr3ghP3QIrfgZNL4El4ZDXwtHnwVHnwvRDcr6tL+00NLWztrGVtdvaWNvYyrrGNp7c2ERv\neu+/h4TBvOmVHFhbxqzacmbVlHFgbXl4XRMe62sUHiITXeyhYGZJ4AXgLKABeBy42N1XZbVZCNwK\nvNHdm8xslrtvG2y8CoUc3GHTn+C5u+H5e8JRSwAzj4QFp8Ohp4fHqplDjqqprZt121tpaOpg2+4u\ntu7uZFtLeHzq5WY6e9N531uriJklAAAN+klEQVSaTHBYfRW1FSVMqyihtjx6rEhlPS+htjxFZWmK\nitIE5SXJ8LwkSVkqQSKhYBEphIkQCqcCX3L3t0SvPwfg7v+S1eabwAvuft1wx6tQGIYda0M4rPs9\nrP0dePRFPmvRnpA45LR9NjUNV2tXL9t2d7J1dxfbWjr7w6OpvYfdnT08sm4HfWmnt8/p7ssfIrkk\nDBJmJMw4eEYlFSXJ0JVmPUbPK0uTlJeErjSVoCyZoDSVoCR6LE0lKI2el6X2DCtJGqlEgmTCSCWM\n1IDXCiaZioYbCqkC1jAXeDnrdQOwZECbIwHM7I+ETUxfcvd7C1hTcZhxOLz270LX1wObVsD6ZfDS\nA/DkT+CxH4V29cfAQSdE3fFw4KuhtHLI0VeXpaiur+aw+uphldPbl6als5ddHSE0Wjp7ae/uo6On\nj87uPtq7e+noSdPR00dHd2/0mKajp5eOqN3y9TtJO6Td6e5NMx57wjLZkHbIxERJMsHsaeWkEkYy\n6jKhksp6ncx+nTBSyQRJAzPDDIzwmIieJxIAFl5HwWjs3T57GDn6Zbff83rffonMOLPaDOzX/5o9\n0+t/f2JP/QPbh88sU9ee+rLbJ7Jq33s+strvU0fm/dnTGvgZ7ds+I/M8s4nTsvtFS3fP68w/eYYN\nGE/28Oz39D/keW+u9w+cXjYzSJoV/AjCQoZCrp9bA/8vp4CFwFJgHvCAmb3a3Zv3GpHZFcAVAAcf\nfDAyAskSmH9S6E7/FPR2wytPwPoHoOFxWHMfPPWzPe1nvQpmvxrqj4KZR0H90eHs6uTo/1RSyQTT\nq0rH/LwJd6erN01XT5quvj66e9Oh60vT0+t09/XRlekX9e/uTdOb9mhNJut5fz+nL72nf8+A13s/\npqPhe14/3bALd3Ccju49d9XL/OGXRv+hu3NshkslDfewzyf7PbB3QEnxmjOtnIc/d2ZBp1HIUGgA\n5me9ngdsytHmEXfvAV4ys+cJIfF4diN3vxa4FsLmo4JVXAxSpXDIqaGDsD9i9ythbWLzirBvYv2D\n8PR/73lPshRmHAEHHBYCYvoCmH5oeKybD6myGGYk/OLKbD4CXSwQQlCGUAprVZmAcg+LOu2+97Bc\n7Qe2IRzFRla/zHhyvX/PeMK001nj9P424Xl6QH17tc+8TtP/3lBGpk2uOvZuH1pnPpzMQ2ibPcwH\nGcaA8fQPHzj+vcazd5uBwzPTyv2e/G1OOLiOQitkKDwOLDSzQ4FXgIuAgUcW3QFcDNxoZjMJm5PW\nFbAmGcgMps0L3TFv3dO/czdsfxG2Pw+Nz0HjC7BjTbgER29n9gigdm5WWGS6Q6BmDlQfGIJIxkVm\nUwpAMufKusjgChYK7t5rZh8BfkXYX3C9u680sy8Dy939zmjYm81sFdAH/L277yhUTTIC5bUw7zWh\ny+YeLrvRtH7fbu1voWXzvuOqnBkComZ26GoPip7P2fNYVQ+JZMFnS0QGp5PXZGz1dITzJ5o3hoBo\n2QK7N4XHzOu2bXuOiMqwRAiGyplQNSM8Vs4Ih9FWzhjwfGY4ciqpTUYiwzURjj6SYlRSEXZS1x+V\nv01fL7Q1RiGxeU9YtGyB9p3h0uFbng6XEu9szj+e8mkDwuOA8LpiehhWUQflddHjtPC8fJrWSEQG\noVCQ8ZdMQe2c0A2lrxc6doaAaN8RAqNt+57waN8RXjdvDDvJ27ZDeohbmJbVRmGRFRSZAOkPkaz+\nZbVhc1pZLZRW5T5eUGSKUCjIxJZMQfWs0A2HO/S0Q0dzWMvo3DX0853r9jzvaR98/JaMAqIGSmug\nrBpKq6PHzOuqqF9N1rDqvZ9n2u/Hob4ihaC/SJlazKIv5SqYNnfk7+/tDoHR2RwFxS7o2hX12w1d\nu/c8drVCdwt0NMGul6PXbaHfwH0m+aTKo8Co2jdEcobJIEFTWq2Qkf2mvyCRbKlSqK4P3Wi5hx3u\n3a3Q1RIeu9v2hEhXazQset0/LGrfvgOaNkQBE/Ub7jncqYooYKpzrMnkWVvpX7vJel5SGR6Tpdpc\nVmQUCiJjzSxcLqS0cvibvQaT2SSWCY7+QMkKnZzDose2xnDIcHa/4YaMJbNCohJKqqLHiqznlXna\nZPrnaVtSSXR9D5lAFAoiE132JjEO3P/xpdMhZPKFSXcrdLdDT1v02B7WWno69jzvboPWxn3bjPSq\nVKmKPUFSUpEnVAYLpEHaJku0ljMKCgWRYpNIhM1EZdVQM4bjdQ9nu/d0RCHSnvWYL2QG9ovatm7d\n0z/Tr69rZPXss5aT6Sqy1mAG6TfwPQP7pcqn5JqOQkFExoZZ9OVZMerLsg+qrzcrJHKFTXbI5Aqg\naHhPRzikuad973593SOvqSRrzSUz73nDJRqeqtjzvP91+Z6gGTgsVTauazwKBRGZHJIpSEbnjBRC\nduhkAiN7bWXY/TqiI9JeGdBmFJvXAIjCNlUOp3wIXv/3Yz3ne1EoiIhA4UPHPayNZNZaejNrL50h\nOHqjx8Fezzq6MLVlUSiIiIwHs7ApKFUWzpSfoKbeXhIRERk1hYKIiPRTKIiISD+FgoiI9FMoiIhI\nP4WCiIj0UyiIiEg/hYKIiPQz99Gcdh0fM2sENozy7TOB7WNYzkRWLPOq+Zx6imVex3s+D3H3IW8U\nMulCYX+Y2XJ3Xxx3HeOhWOZV8zn1FMu8TtT51OYjERHpp1AQEZF+xRYK18ZdwDgqlnnVfE49xTKv\nE3I+i2qfgoiIDK7Y1hRERGQQCgUREelXNKFgZmeb2fNmtsbMroy7nrFkZuvN7BkzW2Fmy6N+B5jZ\nfWb2YvQ4Pe46R8PMrjezbWb2bFa/nPNmwXejZfy0mZ0YX+Ujk2c+v2Rmr0TLdYWZnZs17HPRfD5v\nZm+Jp+qRM7P5Zna/ma02s5Vm9rGo/5RapoPM58Rfpu4+5TsgCawFDgNKgaeARXHXNYbztx6YOaDf\nN4Ero+dXAt+Iu85RztsZwInAs0PNG3Au8EvAgFOAR+Oufz/n80vAp3O0XRT9DZcBh0Z/28m452GY\n8zkHODF6XgO8EM3PlFqmg8znhF+mxbKmcDKwxt3XuXs3cAtwfsw1Fdr5wI+j5z8G3h5jLaPm7suA\nnQN655u384GfePAIUGdmc8an0v2TZz7zOR+4xd273P0lYA3hb3zCc/fN7v5k9LwFWA3MZYot00Hm\nM58Js0yLJRTmAi9nvW5g8AU02TjwazN7wsyuiPod6O6bIfyBArNiq27s5Zu3qbicPxJtNrk+axPg\nlJhPM1sAnAA8yhRepgPmEyb4Mi2WULAc/abSsbinufuJwDnAh83sjLgLislUW84/BA4Hjgc2A/8W\n9Z/082lm1cBtwMfdffdgTXP0mzTzmmM+J/wyLZZQaADmZ72eB2yKqZYx5+6bosdtwM8Jq51bM6vZ\n0eO2+Cocc/nmbUotZ3ff6u597p4G/oM9mxMm9XyaWQnhi/Imd7896j3llmmu+ZwMy7RYQuFxYKGZ\nHWpmpcBFwJ0x1zQmzKzKzGoyz4E3A88S5u8vo2Z/CfwingoLIt+83Qm8Lzpi5RRgV2aTxGQ0YNv5\nOwjLFcJ8XmRmZWZ2KLAQeGy86xsNMzPgP4HV7n511qAptUzzzeekWKZx76Ufr45wFMMLhL36/xh3\nPWM4X4cRjlp4CliZmTdgBvBb4MXo8YC4ax3l/N1MWM3uIfyaen++eSOsgn8/WsbPAIvjrn8/5/O/\novl4mvClMSer/T9G8/k8cE7c9Y9gPl9H2CzyNLAi6s6dast0kPmc8MtUl7kQEZF+xbL5SEREhkGh\nICIi/RQKIiLST6EgIiL9FAoiItJPoSAiIv0UCiLDYGbHD7jM8dvG6hLsZvZxM6sci3GJ7C+dpyAy\nDGZ2GeHEqY8UYNzro3FvH8F7ku7eN9a1iGhNQaYUM1sQ3djkP6Kbm/zazCrytD3czO6Nri77gJkd\nHfV/l5k9a2ZPmdmy6NIoXwbeHd0Y5d1mdpmZfS9qf6OZ/TC6qco6M3t9dAXM1WZ2Y9b0fmhmy6O6\n/jnq91HgIOB+M7s/6nexhZsmPWtm38h6f6uZfdnMHgVONbOvm9mq6Iqb/1qYT1SKTtyng6tTN5Yd\nsADoBY6PXt8KXJKn7W+BhdHzJcDvoufPAHOj53XR42XA97Le2/8auJFwjw4jXBd/N/BnhB9dT2TV\nkrl0QxL4PXBs9Ho90U2SCAGxEagHUsDvgLdHwxy4MDMuwuUQLLtOder2t9OagkxFL7n7iuj5E4Sg\n2Et0SePXAv9jZiuAHxHulgXwR+BGM/sbwhf4cPyfuzshULa6+zMeroS5Mmv6F5rZk8CfgFcR7rY1\n0EnA79290d17gZsId2UD6CNcdRNC8HQC15nZBUD7MOsUGVQq7gJECqAr63kfkGvzUQJodvfjBw5w\n9w+Y2RLgPGCFme3TZpBppgdMPw2koitffho4yd2bos1K5TnGk+u6+hmdHu1HcPdeMzsZOJNw1d+P\nAG8cRp0ig9KaghQlDzc8ecnM3gX9N4g/Lnp+uLs/6u5fALYTrnPfQrjX7mjVAm3ALjM7kHBDpIzs\ncT8KvN7MZppZErgY+MPAkUVrOtPc/R7g44SbtojsN60pSDF7L/BDM/s8UELYL/AU8C0zW0j41f7b\nqN9G4MpoU9O/jHRC7v6Umf2JsDlpHWETVca1wC/NbLO7v8HMPgfcH03/HnfPdS+MGuAXZlYetfvE\nSGsSyUWHpIqISD9tPhIRkX7afCRTnpl9HzhtQO/vuPsNcdQjMpFp85GIiPTT5iMREemnUBARkX4K\nBRER6adQEBGRfv8fygqOt5Iqw2sAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x829eef0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cvresult = pd.DataFrame.from_csv('1_nestimators.csv')\n",
    "        \n",
    "# plot\n",
    "test_means = cvresult['test-mlogloss-mean']\n",
    "test_stds = cvresult['test-mlogloss-std'] \n",
    "        \n",
    "train_means = cvresult['train-mlogloss-mean']\n",
    "train_stds = cvresult['train-mlogloss-std'] \n",
    "\n",
    "x_axis = range(0, cvresult.shape[0])\n",
    "        \n",
    "pyplot.errorbar(x_axis, test_means, yerr=test_stds ,label='Test')\n",
    "pyplot.errorbar(x_axis, train_means, yerr=train_stds ,label='Train')\n",
    "pyplot.title(\"XGBoost n_estimators vs Log Loss\")\n",
    "pyplot.xlabel( 'n_estimators' )\n",
    "pyplot.ylabel( 'Log Loss' )\n",
    "pyplot.savefig( 'n_estimators4_1.png' )\n",
    "\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 通过上图分析我们取树的弱分类器数目取150 即n_estimators=150"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第二步：调整树的参数：max_depth & min_child_weight"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'max_depth': range(3, 10, 2), 'min_child_weight': range(1, 6, 2)}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#max_depth 建议3-10， min_child_weight=1／sqrt(ratio_rare_event) =5.5\n",
    "max_depth = range(3,10,2)\n",
    "min_child_weight = range(1,6,2)\n",
    "param_test2_1 = dict(max_depth=max_depth, min_child_weight=min_child_weight)\n",
    "param_test2_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Anaconda\\envs\\python3\\lib\\site-packages\\sklearn\\model_selection\\_search.py:761: DeprecationWarning: The grid_scores_ attribute was deprecated in version 0.18 in favor of the more elaborate cv_results_ attribute. The grid_scores_ attribute will not be available from 0.20\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "([mean: -0.61024, std: 0.00322, params: {'max_depth': 3, 'min_child_weight': 1},\n",
       "  mean: -0.61073, std: 0.00305, params: {'max_depth': 3, 'min_child_weight': 3},\n",
       "  mean: -0.61027, std: 0.00328, params: {'max_depth': 3, 'min_child_weight': 5},\n",
       "  mean: -0.59268, std: 0.00417, params: {'max_depth': 5, 'min_child_weight': 1},\n",
       "  mean: -0.59344, std: 0.00362, params: {'max_depth': 5, 'min_child_weight': 3},\n",
       "  mean: -0.59340, std: 0.00342, params: {'max_depth': 5, 'min_child_weight': 5},\n",
       "  mean: -0.58857, std: 0.00365, params: {'max_depth': 7, 'min_child_weight': 1},\n",
       "  mean: -0.58881, std: 0.00402, params: {'max_depth': 7, 'min_child_weight': 3},\n",
       "  mean: -0.58806, std: 0.00393, params: {'max_depth': 7, 'min_child_weight': 5},\n",
       "  mean: -0.59395, std: 0.00343, params: {'max_depth': 9, 'min_child_weight': 1},\n",
       "  mean: -0.59379, std: 0.00575, params: {'max_depth': 9, 'min_child_weight': 3},\n",
       "  mean: -0.59142, std: 0.00402, params: {'max_depth': 9, 'min_child_weight': 5}],\n",
       " {'max_depth': 7, 'min_child_weight': 5},\n",
       " -0.58806154462978555)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb2_1 = XGBClassifier(\n",
    "        learning_rate =0.1,\n",
    "        n_estimators=150,  #第一轮参数调整得到的n_estimators最优值\n",
    "        max_depth=5,\n",
    "        min_child_weight=1,\n",
    "        gamma=0,\n",
    "        subsample=0.3,\n",
    "        colsample_bytree=0.8,\n",
    "        colsample_bylevel = 0.7,\n",
    "        objective= 'multi:softprob',\n",
    "        seed=3)\n",
    "\n",
    "#采用GridSearchCV来查看最优的树的深度和子节点权重\n",
    "gsearch2_1 = GridSearchCV(xgb2_1, param_grid = param_test2_1, scoring='neg_log_loss',n_jobs=-1, cv=kfold)\n",
    "gsearch2_1.fit(X_train , y_train)\n",
    "\n",
    "gsearch2_1.grid_scores_, gsearch2_1.best_params_,     gsearch2_1.best_score_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Anaconda\\envs\\python3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda\\envs\\python3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda\\envs\\python3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda\\envs\\python3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda\\envs\\python3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split3_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda\\envs\\python3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split4_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda\\envs\\python3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'mean_fit_time': array([ 127.49679995,  129.23680005,  129.48020005,  180.86600003,\n",
       "         180.66220007,  180.72120004,  234.34259992,  234.53239994,\n",
       "         233.22479987,  289.61059999,  288.77880006,  262.054     ]),\n",
       " 'mean_score_time': array([ 0.31879997,  0.31819997,  0.31700001,  0.50220008,  0.4987999 ,\n",
       "         0.50219994,  0.69200006,  0.6848    ,  0.66860008,  1.01480002,\n",
       "         0.94379992,  0.74120002]),\n",
       " 'mean_test_score': array([-0.61023975, -0.61072822, -0.6102707 , -0.59267592, -0.59344136,\n",
       "        -0.59339503, -0.58857442, -0.58881437, -0.58806154, -0.59394802,\n",
       "        -0.59379317, -0.59141987]),\n",
       " 'mean_train_score': array([-0.5940347 , -0.59474224, -0.59476356, -0.5383926 , -0.54210454,\n",
       "        -0.54427819, -0.45685637, -0.4720282 , -0.48134492, -0.35423181,\n",
       "        -0.38926249, -0.41111103]),\n",
       " 'param_max_depth': masked_array(data = [3 3 3 5 5 5 7 7 7 9 9 9],\n",
       "              mask = [False False False False False False False False False False False False],\n",
       "        fill_value = ?),\n",
       " 'param_min_child_weight': masked_array(data = [1 3 5 1 3 5 1 3 5 1 3 5],\n",
       "              mask = [False False False False False False False False False False False False],\n",
       "        fill_value = ?),\n",
       " 'params': [{'max_depth': 3, 'min_child_weight': 1},\n",
       "  {'max_depth': 3, 'min_child_weight': 3},\n",
       "  {'max_depth': 3, 'min_child_weight': 5},\n",
       "  {'max_depth': 5, 'min_child_weight': 1},\n",
       "  {'max_depth': 5, 'min_child_weight': 3},\n",
       "  {'max_depth': 5, 'min_child_weight': 5},\n",
       "  {'max_depth': 7, 'min_child_weight': 1},\n",
       "  {'max_depth': 7, 'min_child_weight': 3},\n",
       "  {'max_depth': 7, 'min_child_weight': 5},\n",
       "  {'max_depth': 9, 'min_child_weight': 1},\n",
       "  {'max_depth': 9, 'min_child_weight': 3},\n",
       "  {'max_depth': 9, 'min_child_weight': 5}],\n",
       " 'rank_test_score': array([10, 12, 11,  5,  7,  6,  2,  3,  1,  9,  8,  4]),\n",
       " 'split0_test_score': array([-0.6056791 , -0.60634922, -0.60521926, -0.58571039, -0.58690285,\n",
       "        -0.58740977, -0.58222782, -0.58188046, -0.58196081, -0.58851782,\n",
       "        -0.58513283, -0.58603009]),\n",
       " 'split0_train_score': array([-0.59541283, -0.59618696, -0.59606585, -0.53898363, -0.5434112 ,\n",
       "        -0.54596459, -0.45896873, -0.47478163, -0.48413391, -0.35435098,\n",
       "        -0.3880158 , -0.41023122]),\n",
       " 'split1_test_score': array([-0.6093741 , -0.60933562, -0.60915519, -0.59064472, -0.59248192,\n",
       "        -0.59311902, -0.58860778, -0.58823148, -0.58626464, -0.59264089,\n",
       "        -0.59013208, -0.58949585]),\n",
       " 'split1_train_score': array([-0.59430718, -0.59457863, -0.5948409 , -0.53897569, -0.54274104,\n",
       "        -0.54411906, -0.45596259, -0.47085417, -0.48026515, -0.35234095,\n",
       "        -0.38954539, -0.41074078]),\n",
       " 'split2_test_score': array([-0.61004223, -0.61081644, -0.61000821, -0.59344751, -0.59464947,\n",
       "        -0.59300848, -0.58974921, -0.5909466 , -0.5886792 , -0.59621548,\n",
       "        -0.59637704, -0.59109134]),\n",
       " 'split2_train_score': array([-0.59408801, -0.59488787, -0.59415172, -0.5396444 , -0.54302401,\n",
       "        -0.54439026, -0.45730247, -0.47120076, -0.48043474, -0.35274508,\n",
       "        -0.38833518, -0.40944066]),\n",
       " 'split3_test_score': array([-0.61572916, -0.61571748, -0.61524443, -0.5966684 , -0.59612103,\n",
       "        -0.59705292, -0.58871772, -0.5889393 , -0.58947591, -0.59368728,\n",
       "        -0.59527631, -0.59218129]),\n",
       " 'split3_train_score': array([-0.59402399, -0.59461491, -0.59489532, -0.5373802 , -0.54064286,\n",
       "        -0.54342327, -0.4560615 , -0.47073997, -0.48067792, -0.35406392,\n",
       "        -0.3891421 , -0.41228294]),\n",
       " 'split4_test_score': array([-0.61037421, -0.61142258, -0.61172685, -0.59690986, -0.59705263,\n",
       "        -0.59638586, -0.59357107, -0.59407559, -0.59392895, -0.59868006,\n",
       "        -0.60205008, -0.59830286]),\n",
       " 'split4_train_score': array([-0.5923415 , -0.59344282, -0.59386401, -0.53697906, -0.54070358,\n",
       "        -0.54349379, -0.45598658, -0.47256447, -0.48121288, -0.35765813,\n",
       "        -0.39127399, -0.41285952]),\n",
       " 'std_fit_time': array([  2.13228667,   1.19951802,   0.917856  ,   1.22391287,\n",
       "          1.58738922,   1.71164725,   1.01051404,   0.75393619,\n",
       "          1.17797151,   3.1964213 ,   2.42127967,  23.93761126]),\n",
       " 'std_score_time': array([ 0.00584475,  0.00391928,  0.00340598,  0.00664529,  0.00381578,\n",
       "         0.01083325,  0.01181526,  0.00941066,  0.00615133,  0.03579602,\n",
       "         0.04075975,  0.10760925]),\n",
       " 'std_test_score': array([ 0.00321692,  0.0030496 ,  0.00327693,  0.00417387,  0.00361536,\n",
       "         0.00341687,  0.00365087,  0.00401641,  0.00393066,  0.00342911,\n",
       "         0.00575234,  0.00402008]),\n",
       " 'std_train_score': array([ 0.00098421,  0.00087635,  0.00076195,  0.00102756,  0.00118803,\n",
       "         0.00091957,  0.00117032,  0.0015226 ,  0.00143074,  0.00187445,\n",
       "         0.00114497,  0.00127529])}"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gsearch2_1.cv_results_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: -0.588062 using {'max_depth': 7, 'min_child_weight': 5}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Anaconda\\envs\\python3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda\\envs\\python3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda\\envs\\python3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda\\envs\\python3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda\\envs\\python3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda\\envs\\python3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split3_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\Anaconda\\envs\\python3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split4_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAELCAYAAADz6wBxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xt8XHWd//HXZy5JmqaUXuklKa1c\nK22pEAvKgsXd0qIsWHEREBZYBFflx7r+QMB1ZWERdX38BC8VBQVclYugxeoWaMFCFRd6YVMsRWwF\ntg0FWgotbdommZnP7485M53MJZnkZHJp38/HYx45c873nPnOSXLe53zP5WvujoiISE9F+rsCIiIy\nuClIREQkFAWJiIiEoiAREZFQFCQiIhKKgkREREJRkIiISCgKEhERCUVBIiIiocT6uwJ9YfTo0T55\n8uT+roaIyKCyevXqN919TFflDoggmTx5MqtWrervaoiIDCpm9r/llFPTloiIhKIgERGRUBQkIiIS\nioJERERCUZCIiEgoChIREQlFQSIiIqEcEPeRiFRSKuW0JVO0JlK0JVK0JYOfiRStieS+4Q7j0z9T\n7lRFI8RjRlU0SlUsQlUsQjxqVMciVEWjwbRIdlp2OBohGjHMrL9XgRzgFCQy6Lh7h411W94GutMN\nerLjhnxf2WSH5bW2pzqGQ2Z8TjBkxrUnvd/WhRnZYKmORYjnhEw8J3yy06KZoNo3PhNcmSDLhlgs\np2w0L8TypnX47FiEmALugKIgkS65O4mUF2y4W/P2uPM36Pl74cU26JkymQ13W4kNfv5efW8ptqef\nuxGtikYYVhOjOhahOhYtujGtzilb3WF8tKBsVTRCTTyS3WhHDNpz123uukmmaM9bV6WmtSZStOdO\ny1t/La2JbNn2pOdMS6bfJ1MkU70XiGYQj0aozguubMjFcqdZMC2zvqwg8PKPxPKnVQfLzJ1Wnfd5\n6XBVwFWCgmSASuZsXFqDveX8vePcjXPxvfDCDX+pjXlbh435vo1MZq/ce2kbE4tY4YYhZyNdHY1Q\nWxXj4MyGOV5swx0Nmn06bmTyy2aahootoyrYk45EtFHJSKac9rwmupJhlRnOKZ8/rTW/bHZZnt1p\n2N2WyAZbW7Lwb7g9mSLRiwEHhTsP+5oOo1RFO/59lhNk+UeC+UGWPy3387MhGo0M6oCraJCY2Tzg\nW0AU+KG7f61ImXOAfwMcWOPu5wfjHwFOBH7v7mfklJ8C3AeMBJ4FLnT3tkrU//Ude9mxpz3b9FHQ\nHFKi+aRwL7xI2WSK1vZk0b3N1kTv7R2akbPRjWb/uKvz/jnqamLBxnjfXnSpPe2Oe+/Rgj34/L36\n6py2/6g23ANWNGJEI1Fq4tH+rkoHmYBry/u/yoZeN47c0uM8+z/dnvCiQbm3PcXOvYkO/8P5dahE\nwMVzgiw3gPKbDrMhVuToLBtiwbizZk7g4NqqXq1rvooFiZlFgQXAHKAZWGlmi9x9XU6ZI4DrgJPc\n/W0zG5uziG8AtcCn8hb9deAWd7/PzL4PXArcVonvcO0vn+OJF7d2e77cX3Cx5o+qWIThVXGq6qqp\njueVzdlAd5i/RLNLwV563ni1VctgN1ADLnORRadBlmlKzGlVyB6B5TQrdueobldrouiRW6lzdn91\nxOjBGyTALGCDu78EYGb3AWcB63LKXAYscPe3Adx9S2aCuz9uZrNzF2jpLeIHgfODUT8mfTRTkSD5\n1CmH8XfHNxTd2y61QR/sh6giUp5IxKgZgAGXfzHK8CHxin9mJYNkIrAp530zcEJemSMBzOwp0s1f\n/+buj3SyzFHAdndP5CxzYu9Ut9D7DhtVqUWLiFSEmQUtFH0XcJUMkmK75fmNijHgCGA2UA/8zsym\nufv2EMtMFzS7HLgcYNKkSeXUV0REeqCSd7Y3Aw057+uBzUXK/Mrd2939ZeBF0sFSypvAwWaWCcBi\nywTA3W9390Z3bxwzpssOvkREpIcqGSQrgSPMbIqZVQHnAovyyjwEnApgZqNJN3W9VGqB7u7AMuBj\nwaiLgF/1cr1FRKQbKhYkwXmMK4BHgReAn7v782Z2o5mdGRR7FNhmZutIB8TV7r4NwMx+BzwA/LWZ\nNZvZ3GCea4DPm9kG0udMflSp7yAiIl0z7607zQawxsZGV5/tIiLdY2ar3b2xq3J6+q+IiISiIBER\nkVAUJCIiEoqCREREQlGQiIhIKAoSEREJRUEiIiKhKEhERCQUBYmIiISiIBERkVAUJCIiEoqCRERE\nQlGQiIhIKAoSEREJRUEiIiKhKEhERCQUBYmIiISiIBERkVAUJCIiEoqCREREQlGQiIhIKAoSEREJ\nRUEiIiKhKEhERCQUBYmIiISiIBERkVAUJCIiEoqCREREQlGQiIhIKAoSEREJRUEiIiKhKEhERCQU\nBYmIiISiIBERkVAUJCIiEkpFg8TM5pnZi2a2wcyuLVHmHDNbZ2bPm9k9OeMvMrP1weuinPFPBMts\nCl5jK/kdRESkc7FKLdjMosACYA7QDKw0s0Xuvi6nzBHAdcBJ7v52JhTMbCRwPdAIOLA6mPftYNZP\nuPuqStVdRETKV8kjklnABnd/yd3bgPuAs/LKXAYsyASEu28Jxs8Flrr7W8G0pcC8CtZVRER6qJJB\nMhHYlPO+ORiX60jgSDN7ysyeNrN5Zc57V9Cs9a9mZsU+3MwuN7NVZrZq69at4b6JiIiUVMkgKbaB\n97z3MeAIYDZwHvBDMzu4i3k/4e7TgZOD14XFPtzdb3f3RndvHDNmTA+qLyIi5ahkkDQDDTnv64HN\nRcr8yt3b3f1l4EXSwVJyXnd/Nfi5E7iHdBOaiIj0k0oGyUrgCDObYmZVwLnAorwyDwGnApjZaNJN\nXS8BjwKnmdkIMxsBnAY8amaxoBxmFgfOANZW8DuIiEgXKnbVlrsnzOwK0qEQBe509+fN7EZglbsv\nYl9grAOSwNXuvg3AzP6ddBgB3Ojub5nZUNKBEg+W+RhwR6W+g4iIdM3c809b7H8aGxt91SpdLSwi\n0h1mttrdG7sqpzvbRUQkFAWJiIiEoiAREZFQFCQiIhKKgkREREJRkIiISCgKEhERCUVBIiIioShI\nREQkFAWJiIiEUrFnbYnI4NTe3k5zczN79+7t76pIH6mpqaG+vp54PN6j+RUkItJBc3Mzw4YNY/Lk\nyZToN072I+7Otm3baG5uZsqUKT1ahpq2RKSDvXv3MmrUKIXIAcLMGDVqVKgjUAWJiBRQiBxYwv6+\nFSQiIhKKgkREBpTt27fzve99r0fz3nrrrezevbuXa9Q7Zs+eTU/7RXrooYdYt25dt5a1d+9eZs2a\nxbHHHssxxxzD9ddf36PPLoeCREQGlP01SMLID5JyVFdX89vf/pY1a9bQ1NTEI488wtNPP12R+umq\nLREp6YZfP8+6ze/06jLfPeEgrv/bY0pOv/baa/nLX/7CzJkzmTNnDmPHjuXnP/85ra2tzJ8/nxtu\nuIGWlhbOOeccmpubSSaT/Ou//itvvPEGmzdv5tRTT2X06NEsW7as6PLr6ur47Gc/y2OPPcaIESO4\n+eab+cIXvsDGjRu59dZbOfPMM3nllVe48MILaWlpAeC73/0u73//+1m4cCELFixg6dKlvP7663zg\nAx9g+fLljBs3ruBz9uzZwyWXXMK6deuYOnUqe/bsyU5bsmQJ119/Pa2trRx22GHcdddd1NXVMXny\nZD7+8Y9n637PPfewZcsWFi1axJNPPslNN93EL37xCwAeeOABPvOZz7B9+3Z+9KMfcfLJJ3f4fDOj\nrq4OSF/S3d7eXrFzXzoiEZEB5Wtf+xqHHXYYTU1NzJkzh/Xr17NixQqamppYvXo1y5cv55FHHmHC\nhAmsWbOGtWvXMm/ePK688komTJjAsmXLSoYIQEtLC7Nnz2b16tUMGzaML33pSyxdupSFCxfy5S9/\nGYCxY8eydOlSnn32We6//36uvPJKAObPn8+4ceNYsGABl112GTfccEPREAG47bbbqK2t5bnnnuNf\n/uVfWL16NQBvvvkmN910E4899hjPPvssjY2NfPOb38zOd9BBB7FixQquuOIKPve5z/H+97+fM888\nk2984xs0NTVx2GGHAZBIJFixYgW33norN9xwAwCbN2/mQx/6UHZZyWSSmTNnMnbsWObMmcMJJ5wQ\n4jdTmo5IRKSkzo4c+sKSJUtYsmQJ73nPewDYtWsX69ev5+STT+aqq67immuu4YwzzijYG+9MVVUV\n8+bNA2D69OlUV1cTj8eZPn06r7zyCpDeg7/iiitoamoiGo3y5z//OTv/d77zHaZNm8aJJ57Ieeed\nV/Jzli9fng2gGTNmMGPGDACefvpp1q1bx0knnQRAW1sb73vf+7LzZZZ53nnn8c///M8ll//Rj34U\ngOOPPz5b7wkTJrB48eJsmWg0SlNTE9u3b2f+/PmsXbuWadOmlbWeukNBIiIDlrtz3XXX8alPfapg\n2urVq1m8eDHXXXcdp512WvZooivxeDzbxBOJRKiurs4OJxIJAG655RYOOeQQ1qxZQyqVoqamJjv/\nq6++SiQS4Y033iCVShGJlG7YKdaU5O7MmTOHe++9t8t5OmuKytQ7Go1m613KwQcfzOzZs3nkkUcq\nEiRq2hKRAWXYsGHs3LkTgLlz53LnnXeya9cuIL0R37JlC5s3b6a2tpYLLriAq666imeffbZg3jB2\n7NjB+PHjiUQi/OQnPyGZTALp5qRLLrmEe+65h6lTp3Zoksp3yimn8LOf/QyAtWvX8txzzwFw4okn\n8tRTT7FhwwYAdu/e3eGI5/7778/+zByp9OR7bd26le3btwPp8zWPPfYYRx99dLeWUS4dkYjIgDJq\n1ChOOukkpk2bxumnn87555+f3aDW1dXx05/+lA0bNnD11VcTiUSIx+PcdtttAFx++eWcfvrpjB8/\nvtPzJF35zGc+w9lnn80DDzzAqaeeytChQwG4+eabOfnkkzn55JOZOXMm733ve/nwhz/M1KlTC5bx\n6U9/mksuuYQZM2Ywc+ZMZs2aBcCYMWO4++67Oe+882htbQXgpptu4sgjjwSgtbWVE044gVQqlT1q\nOffcc7nsssv49re/zYMPPliy3ps3b+aTn/wkixcv5rXXXuOiiy4imUySSqU455xzOOOMM3q8Tjpj\n7t55AbPDgGZ3bzWz2cAM4D/dfXtFalQBjY2N3tPrt0UONC+88ELRDaNU3uTJk1m1ahWjR4/u888u\n9ns3s9Xu3tjVvOU0bf0CSJrZ4cCPgCnAPT2pqIiI7H/KadpKuXvCzOYDt7r7d8zsfypdMRGRME44\n4YRs01HGT37yE6ZPn96rn/Poo49yzTXXdBg3ZcoUFi5c2O1lZa6+GmzKCZJ2MzsPuAj422Bczx5a\nLyLSR5555pk++Zy5c+cyd+7cPvmsgaqcpq1LgPcBX3H3l81sCvDTylZLREQGiy6PSNx9HXAlgJmN\nAIa5+9cqXTERERkcujwiMbMnzOwgMxsJrAHuMrPSF0+LiMgBpZymreHu/g7wUeAudz8e+JvKVktE\nRAaLcoIkZmbjgXOA31S4PiJygNtfHyPf1/2RQPq+lOnTpzNz5kwaG7u8HaTHygmSG4FHgb+4+0oz\nexewvmI1EpED2v4aJGH0pD+SjGXLltHU1NTjECtHOSfbHwAeyHn/EnB2xWokIgPHw9fC63/s3WWO\nmw6nl75eR/2R9E5/JH2pnJPt9Wa20My2mNkbZvYLM6svZ+FmNs/MXjSzDWZ2bYky55jZOjN73szu\nyRl/kZmtD14X5Yw/3sz+GCzz21apnlpEpF+oP5Le64/EzDjttNM4/vjjuf3220P8VjpXzg2Jd5F+\nJMrfBe8vCMbN6WwmM4sCC4JyzcBKM1sUXE6cKXMEcB1wkru/bWZjg/EjgeuBRsCB1cG8bwO3AZcD\nTwOLgXnAw+V9XRHplk6OHPqC+iMJ1x/JU089xYQJE9iyZQtz5szh6KOP5pRTTilrPXVHOUEyxt3v\nynl/t5l9roz5ZgEbgqYwzOw+4Cwgt6HvMmBBEBC4+5Zg/Fxgqbu/Fcy7FJhnZk8AB7n7fwfj/xP4\nCAoSkf2S+iMJ1x/JhAkTgPQR1vz581mxYkVFgqSck+1vmtkFZhYNXhcA28qYbyKwKed9czAu15HA\nkWb2lJk9bWbzuph3YjDc2TJFZBBTfyS90x9JS0tLdp6WlhaWLFlSkU6toLwjkn8AvgvcQrqZ6Q+k\nH5vSlWJRmv/M+hhwBDAbqAd+Z2bTOpm3nGWmP9zsctJNYEyaNKmM6orIQKD+SHqnP5I33niD+fPn\nA+kAPP/887NNer2ty/5Iis5k9jl3v7WLMu8D/s3d5wbvrwNw96/mlPk+8LS73x28fxy4FjgcmO3u\nnwrG/wB4Ingtc/ejg/Hn5ZYrRf2RiJRP/ZH0n/25P5JiPl9GmZXAEWY2xcyqgHOBRXllHgJOBTCz\n0aSbul4ifd/KaWY2Ini+12nAo+7+GrDTzE4Mrtb6e+BXPfwOIiLSC3ra1W6Xl9wGfZhcQToUosCd\n7v68md0IrHL3RewLjHVAErja3bcBmNm/kw4jgBszJ96BTwN3A0NIn2TXiXYRKaD+SPpOT5u2Nrr7\noDnxoKYtkfKpaevAFKZpq+QRiZntpPiJbCN9NCAiIlI6SNx9WF9WREREBqeenmwXEREBFCQiIhKS\ngkREBpT99THyfd0fyYsvvsjMmTOzr4MOOohbb+309r8eU5CIyICyvwZJGD3pj+Soo46iqakp+9Tk\n2tra7J3uva3L+0hKXL21A1gF/N/MQxlFZP/z9RVf509v/alXl3n0yKO5ZtY1JaerP5Le74/k8ccf\n57DDDuPQQw/t+hfUA+UckXwTuJr0wxHrgauAO4D7gDsrUisROWCpP5Le648k47777uv0kfdhlXNn\n+zx3PyHn/e1m9rS732hmX6xUxUSk/3V25NAX1B9JuP5IMstftGgRX/3qV/MX0WvKCZKUmZ0DZB45\n+bGcad2/LV5EpEzqjyRcfyQADz/8MMcddxyHHHJIyTJhldO09QngQmBL8LoQuMDMhgBXVKxmInJA\nUn8kvdMfSca9995b0WYtKOOIJDiZ/rclJv++d6sjIgc69UfSO/2RQDqkli5dyg9+8IMer4tydPnQ\nRjOrB74DnES6Kev3wD+5e3OnMw4gemijSPn00Mb+sz/3R3IX6X5EJpC+cuvXwTgREZGyTraPcffc\n4LjbzD5XqQqJiPQG9UfSd8oJkjfN7AIgc4nBecC2ylVJRCS8Z555pk8+Z+7cucydO7dPPmugKqdp\n6x+Ac4DXgddIX/57SSUrJSIig0eXQeLuG939THcf4+5j3f0jwEf7oG4iIjII9PShjZ/v1VqIiMig\n1dMgKX27pYiIHFB6GiR6NIqIVMT++hj5vu6PBOBb3/oW06ZN45hjjqlYXyTQSZCY2U4ze6fIayfp\ne0pERHrd/hokYfSkP5K1a9dyxx13sGLFCtasWcNvfvMb1q9fX5H6lbz8192HVeQTRWTQeP3mm2l9\noXf7I6meejTjvlj6weHqj6R3+iN54YUXOPHEE6mtrQXgAx/4AAsXLuQLX/hCN35b5VEPiSIyoKg/\nkt7pj2TatGksX76cbdu2sXv3bhYvXsymTZtC/naKK+eGRBE5QHV25NAX1B9Jz/sjmTp1Ktdccw1z\n5syhrq6OY489llisMpt8BYmIDFjqjyRcfySXXnopl156KQBf/OIXqa+vL7m8MNS0JSIDivoj6b3+\nSLZs2QLAxo0b+eUvf1mxfkl0RCIiA4r6I+m9/kjOPvtstm3bRjweZ8GCBYwYMaLH66QzXfZHsj9Q\nfyQi5VN/JP1nf+6PREREpCQ1bYnIfkn9kfQdBYmIFHD3Tq8YGgzUH0n5wp7iUNOWiHRQU1PDtm3b\nQm9cZHBwd7Zt29bhEufu0hGJiHRQX19Pc3MzW7du7e+qSB+pqakJdY+JgkREOojH40yZMqW/qyGD\nSEWbtsxsnpm9aGYbzOzaItMvNrOtZtYUvD6ZM+3rZrY2eH08Z/zdZvZyzjwzK/kdRESkcxU7IjGz\nKLAAmAM0AyvNbJG75z8L+X53vyJv3g8DxwEzgWrgSTN72N3fCYpc7e6l78oREZE+U8kjklnABnd/\nyd3bgPuAs8qc993Ak+6ecPcWYA0wr0L1FBGRECoZJBOB3GcWNwfj8p1tZs+Z2YNm1hCMWwOcbma1\nZjYaOBVoyJnnK8E8t5hZdUVqLyIiZalkkBS7CD3/esJfA5PdfQbwGPBjAHdfAiwG/gDcC/w3kHm8\n5XXA0cB7gZHANRRhZpeb2SozW6WrT0REKqeSQdJMx6OIemBzbgF33+bumVtP7wCOz5n2FXef6e5z\nSIfS+mD8a57WCtxFugmtgLvf7u6N7t44ZsyYXvtSIiLSUSWDZCVwhJlNMbMq4FxgUW4BMxuf8/ZM\n4IVgfNTMRgXDM4AZwJLceSx92+1HgLUV/A4iItKFil215e4JM7sCeBSIAne6+/NmdiOwyt0XAVea\n2Zmkm63eAi4OZo8Dvwse0fAOcIG7Z5q2fmZmY0gfpTQB/1ip7yAiIl3TY+RFRKQoPUZeRET6hIJE\nRERC0bO2OpHcuRNPJLBIBKLR9GO1IxGIRNLjMsOD/HHbIiJhKEg68epVV9Hy5PKuC+YGjFnH0CkZ\nQIZZ3vRoJD2u22UNi0QLy0YMrIuywXCHspFIeri7ZaPRgvkskvfdO5mvoGzuOuhO2UgELCib8zvJ\nna+grHYGRHpMQdKJER8/l7q/OhlSSTzlkEqBp/Bk8DOVgtzhlBcvm0rh3s2yKYdkct9wbtlUCk8k\n8IKymel5ZVPFlls4vWPZYHoy2d+/hr6RtyOQHc4PndyQzJa1bFBbNBN65ZW1miHE6ydS1TCJqkkN\nxBsmEa+fSKSqqr/XiEjZFCSdGPbBU/u7CgNCl6GTSoEHAZVy8B6UDUI2XSYTnLkhGsxXbtlg2Zlw\nTgd4senJ0mU7BHkKT3WnbM5377ATEZRNJvB2J/XmNlqefhrfvXvfCjcjNm4cVQ0NxCc17AuZ+gaq\nJjUQHT68//4YRIpQkEiXsnvT0WjR595IOO5Octs22jZuon3TxvTP5k20bdzErieeJPnmmx3KR4YP\np6qhoUO4xIOwiR1ySPoISqQPKUhEwkomoL0F2nZD+25o2xUMB+PaWvKmtwTjdkOiFQNiZsE/o8EE\ngwnArKFgU0m1Jml7ay/tb7fS9tZe2rbtof2tV9nzzF9455G96aOggMUixEcOIT5yCFWjaqkaVUt8\n1BCqRg8lPrKWSDyaPqeX2SUoNpw9X5Q7TInxPR0u9vmUGN/D4X77HvmfVYnvYTlPM+yijhOPh6qh\nVJKCRA4MqVTOxrzUBn5X8Y199mdQJjPcHpRJtnWvLrGa9D92fCjEcs6FuJN9rmnOcMSdGpyaOmAo\n0ODBdPCU074T2ndC2ztO244k7Tt30bZ1F3v+Aqn2jseQsVqnapgTPyiV/lkX/ByWJFoFZoWf32vD\n0j8+uxLGHFnRj1CQyMDhnreR7mwD38n0bLmc5ST2dK8u0SqI10JVHVTVBsNDYeiY9M+qocG4oExm\nOB5Myw7XFi4nEu21VWZAVfDK3+d0d5Jvv037xo20bdpE26ZNtG9M/2zZuJEd6zs+FTsybFhwXmYS\nVQ31xBsaqJo0iaqGBmLjxqWvggvL94UgVHqYygRiZrjgcyrxPTz7tsf1Hd7zvtjLpSCR7nGHRGve\nxrylcMPdoz383XRr7zUS27exzm7Yh0LNwXDQhDI28EXGZY4UooP/X8PMiI0cSWzkSIbMLOyROrVn\nTzpcmptp27gxGzKtL7zAzscfh/b2fYXjcaomTkyHS+YigCBk4vX1RIYMKbdSHZt+ZL8w+P9bpLhE\nW/ENfGft9R325lsKm3Ayw54qvx4Wydszz2y866DukI7jSu7h52zgc/fwY7pENozIkCHUHHkkNUcW\nNnt4Mkni9dfTRzIbN9K+aVNwMcAmdjQ1kdq5s0P52JgxwZFMzpVmDfXEJ00iOmKE7tPZz+mhjf0p\n9yRtr+zh58yTSnT9+bnyN/ZlbeDL2MOP1WgPdD/j7iS3b98XLsEVZpkmtMQbb3QoHxk6NBsyHa40\nmzSJ+LhxWEz7swNVuQ9t1G+wK6lk6T3zck7YFtvDzzTnhDlJm7vRP2hCz5twqmohNiR9A51IGcyM\n2IgRxEaMYMiMGQXTU3v3ppvLNm3Khk3bpo20rl/PrmXL8Nwms1iM+MQJVNXn3TMTHNFEamv78JtJ\nTylIOnPPufDnh7s3T/Yk7dCOe/ZlnaQt0YRTgZO0IpUSqamh+vDDqT788IJpnkqReOONDvfMtG3a\nSPumZvYsXktqx44O5aOjR+87ksmGTPo8TXTUKDWZDRAKks68+ywYf+wBd5JWpFIsEiE+fjzx8ePh\nhMJespM7dnQMmeb0lWYtK1aSWPTrDldKRWprgyvL9h3BZMNm/HgsHu/Lr3ZA01avMzPP6+8aiBxQ\nosOHM2T6cIZMn1YwLdXaSvurr3ZoLmvfuInWl15m15PL8bacpuJolPiECemT/9mwSV9pFq9vIFpX\n2Rv0DjQKEhEZFCLV1VS/611Uv+tdBdM8lSKxdWv6hH9OyLQ1N7P30UdJbt/eoXx05Mice2b2Xc4c\nr68nNmaMmsy6SUEiIoOeRSLEDzmE+CGHUPve9xZMT+7cue8y5pwbM/esXs07//Vf6ScfZJY1ZAhV\n9fX7Qqahft89MxMmYHoycwEFiYjs96LDhjHkmGMYcswxBdO8rY22V18tuDGzfeP/0vLUU/jevfsK\nZ87xTGoovNJs0iSidXV9+K0GDgWJiBzQrKqK6ilTqJ4ypWCau6ebzLI3ZG7M3pi58/HHSb71Vofy\n0YMPLrwxMzg/ExszZr99MrOCRESkBDMjPnYs8bFjqT3++ILpyV27OobMpmbaN21kz5o1vPPIIx06\nhrPq6nQzWV7/MvtDZ2YKEhGRHorW1RGdOpWaqVMLpnl7O+2bN2fDJfcigJZnninszGz8OKoaJnUM\nm+Bn9KCD+vBbdZ+CRESkAiwep+rQQ6k69FDgpA7TinVmlrkxs1hnZtHhwzveM5NzY+ZA6MxMQSIi\n0sfMjNjo0cRGj4bj3lMwPdVqx3DdAAAMTklEQVTSQlvuyf/gxsw9a5/nnSVLIbHvWXpWVUW8vr7j\n5czBlWbx+noi1dUV/z4KEhGRASYydCg1Rx1FzVFHFUzzRIL2114ruDGzbdMmdq9cSSqvyWzKQwuL\nLqc3KUhERAYRi8XSzx9raGDo+ztOK+jMbONG4hPVsZWIiJSpq87MKmX/vKhZRET6jIJERERCUZCI\niEgoChIREQlFQSIiIqEoSEREJJSKBomZzTOzF81sg5ldW2T6xWa21cyagtcnc6Z93czWBq+P54yf\nYmbPmNl6M7vfzAbvk85ERPYDFQsSM4sCC4DTgXcD55nZu4sUvd/dZwavHwbzfhg4DpgJnABcbWaZ\np5Z9HbjF3Y8A3gYurdR3EBGRrlXyiGQWsMHdX3L3NuA+4Kwy53038KS7J9y9BVgDzLN0/5cfBB4M\nyv0Y+Egv11tERLqhkkEyEdiU8745GJfvbDN7zsweNLOGYNwa4HQzqzWz0cCpQAMwCtju7pknlpVa\npoiI9JFKBokVGed5738NTHb3GcBjpI8wcPclwGLgD8C9wH8DiTKXmf5ws8vNbJWZrdq6dWvPvoGI\niHSpkkHSTPooIqMe2JxbwN23uXtr8PYO4PicaV8JzpvMIR0g64E3gYPNLFZqmTnz3+7uje7eOGbM\nmF75QiIiUqiSQbISOCK4yqoKOBdYlFvAzMbnvD0TeCEYHzWzUcHwDGAGsMTdHVgGfCyY5yLgVxX8\nDiIi0oWKPf3X3RNmdgXwKBAF7nT3583sRmCVuy8CrjSzM0k3W70FXBzMHgd+lz63zjvABTnnRa4B\n7jOzm4D/AX5Uqe8gIiJds/RO/v6tsbHRV61a1d/VEBEZVMxstbs3dlVOd7aLiEgoChIREQlFQSIi\nIqEoSEREJBT12S5ShpSnSKaSJD3nldr3M+UpEp7IlssdLiifM4+ZUROtYUhsSPoVH5IdronWEFy5\nKDKgKUgkvcHL3yCmUsU3hkXKFdtQpjxFIpXoslzJeYKfXZXrtG6508v4zM4Coj8YRk0sJ2RiQ6iN\n1XYIm9pYbYfpBa944bjaWC01sRoipgYJ6R0Kkk68vONldrTuKL7BKXfDVGQvtJy91e7M0+WGt4tQ\n8OJPmelXEYsQtWj6FYkSsQgxi6XHR6IdpkUtmB6JFZSLR+JUW3WX8+RPi1qUSKTEZ+bXKSjXaT1z\n58mpZ8pT7E3uZU9iD7sTu9nTvoc9iY6v3Ynd6eGcadv3bi8ol/Rkt9ZxhyOhTMjEuxdMpYIsGolW\n6C9DBiIFSSf+Y+V/8PtXf9/ryzWs0w1T/gan6EYumF5lVZ1v5PKXU2SD2unGushndqdcwWeV+Pz8\nedSk0z3uTnuqvdPwKQinEtO27N7ScRmJPSRS3Tsqq4pUlRc68TKPrHLKxiPxCq1F6SndkNiJtW+u\nZXvrdqIWLXvPtbMNaGa8mhRksMmGVInwyQZU++7CMOvs1b6HtlRbt+oSi8SKh1O8zEDqJMTikbh2\nYnKUe0Oijkg6MW30tP6ugsiAEI/EiVfFOajqoK4Ld1MilWBvYm/pJr28gCr12rF3B68nXu8wbm9y\nb7fqErVoyeApp9mvsyCrjlbvtyGlIBGRfhWLxKirqqOuqq7Xl53yFHsTe4sfHbV3cdSUc/TV0t7C\n1j1bC8p0R8QiJa/QKxZAXQZX3qs/Q0pBIiL7rYhFqI3XUhuv7fVlu3v2Qon84CkWUJ0dVb299+2C\ncSlPdas+pQLmxpNuZGJdZfv/U5CIiPSAmWU31r3N3WlLtRWck8oEVGcXS+S/olb5K+gUJCIiA4yZ\nUR2tpjpazcEc3N/V6ZIuHxIRkVAUJCIiEoqCREREQlGQiIhIKAoSEREJRUEiIiKhKEhERCQUBYmI\niIRyQDz918y2Av/bw9lHA2/2YnV6i+rVPapX96he3bO/1utQdx/TVaEDIkjCMLNV5TxGua+pXt2j\nenWP6tU9B3q91LQlIiKhKEhERCQUBUnXbu/vCpSgenWP6tU9qlf3HND10jkSEREJRUckIiISioIE\nMLM7zWyLma0tMd3M7NtmtsHMnjOz4wZIvWab2Q4zawpeX+6jejWY2TIze8HMnjezfypSps/XWZn1\n6vN1ZmY1ZrbCzNYE9bqhSJlqM7s/WF/PmNnkAVKvi81sa876+mSl65Xz2VEz+x8z+02RaX2+vsqs\nV7+sLzN7xcz+GHzmqiLTK/v/6O4H/As4BTgOWFti+oeAhwEDTgSeGSD1mg38ph/W13jguGB4GPBn\n4N39vc7KrFefr7NgHdQFw3HgGeDEvDKfAb4fDJ8L3D9A6nUx8N2+/hsLPvvzwD3Ffl/9sb7KrFe/\nrC/gFWB0J9Mr+v+oIxLA3ZcDb3VS5CzgPz3taeBgMxs/AOrVL9z9NXd/NhjeCbwA5HcK3efrrMx6\n9blgHewK3saDV/7JybOAHwfDDwJ/bWY2AOrVL8ysHvgw8MMSRfp8fZVZr4Gqov+PCpLyTAQ25bxv\nZgBsoALvC5omHjazY/r6w4MmhfeQ3pvN1a/rrJN6QT+ss6A5pAnYAix195Lry90TwA5g1ACoF8DZ\nQXPIg2bWUOk6BW4FvgCkSkzvl/VVRr2gf9aXA0vMbLWZXV5kekX/HxUk5Sm2pzMQ9tyeJf0Ig2OB\n7wAP9eWHm1kd8Avgc+7+Tv7kIrP0yTrrol79ss7cPenuM4F6YJaZTcsr0i/rq4x6/RqY7O4zgMfY\ndxRQMWZ2BrDF3Vd3VqzIuIqurzLr1efrK3CSux8HnA581sxOyZte0fWlIClPM5C7Z1EPbO6numS5\n+zuZpgl3XwzEzWx0X3y2mcVJb6x/5u6/LFKkX9ZZV/Xqz3UWfOZ24AlgXt6k7PoysxgwnD5s1ixV\nL3ff5u6twds7gOP7oDonAWea2SvAfcAHzeyneWX6Y311Wa9+Wl+4++bg5xZgITArr0hF/x8VJOVZ\nBPx9cOXDicAOd3+tvytlZuMy7cJmNov073NbH3yuAT8CXnD3b5Yo1ufrrJx69cc6M7MxZnZwMDwE\n+BvgT3nFFgEXBcMfA37rwVnS/qxXXjv6maTPO1WUu1/n7vXuPpn0ifTfuvsFecX6fH2VU6/+WF9m\nNtTMhmWGgdOA/Cs9K/r/GOutBQ1mZnYv6at5RptZM3A96ROPuPv3gcWkr3rYAOwGLhkg9foY8Gkz\nSwB7gHMr/c8UOAm4EPhj0L4O8EVgUk7d+mOdlVOv/lhn44Efm1mUdHD93N1/Y2Y3AqvcfRHpAPyJ\nmW0gvWd9boXrVG69rjSzM4FEUK+L+6BeRQ2A9VVOvfpjfR0CLAz2j2LAPe7+iJn9I/TN/6PubBcR\nkVDUtCUiIqEoSEREJBQFiYiIhKIgERGRUBQkIiISioJERERCUZCIDBDBo8B7dJd98PjyCb2xLJHu\nUpCI7B8uBiZ0VUikEhQkInnMbLKZ/cnMfmhma83sZ2b2N2b2lJmtN7NZwesPlu7g6A9mdlQw7+fN\n7M5geHowf22JzxllZkuCZfyAnAfrmdkFlu50qsnMfhDcfY6Z7TKz/2dmz5rZ48FjTj4GNAI/C8oP\nCRbzf4JyfzSzoyu5zuTApiARKe5w4FvADOBo4Hzgr4CrSD925U/AKe7+HuDLwM3BfLcCh5vZfOAu\n4FPuvrvEZ1wP/D5YxiKCR7mY2VTg46Sf6DoTSAKfCOYZCjwbPOn1SeB6d38QWAV8wt1nuvueoOyb\nQbnbgnqLVISetSVS3Mvu/kcAM3seeNzd3cz+CEwm/bTZH5vZEaQfx515BlrKzC4GngN+4O5PdfIZ\npwAfDeb7LzN7Oxj/16SfGrsyeH7SENL9hUC6H4z7g+GfAsWevJyRmbY68zkilaAgESmuNWc4lfM+\nRfr/5t+BZe4+39KdaD2RU/4IYBflnbMo9rA7A37s7tf1cP6MTJ2T6H9dKkhNWyI9Mxx4NRi+ODPS\nzIaTbhI7BRgVnL8oZTlBk5WZnQ6MCMY/DnzMzMYG00aa2aHBtAjpJxhDurnt98HwTtL91Iv0OQWJ\nSM/8B/BVM3sKiOaMvwX4nrv/GbgU+FomEIq4ATjFzJ4l3YfERgB3Xwd8iXTXqc8BS0k/8h2gBTjG\nzFYDHwRuDMbfDXw/72S7SJ/QY+RFBhEz2+Xudf1dD5FcOiIREZFQdEQiUmFmdgnwT3mjn3L3z/ZH\nfUR6m4JERERCUdOWiIiEoiAREZFQFCQiIhKKgkREREJRkIiISCj/H8bh+/Fm7ik0AAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ffbef0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# summarize results\n",
    "print(\"Best: %f using %s\" % (gsearch2_1.best_score_, gsearch2_1.best_params_))\n",
    "test_means = gsearch2_1.cv_results_[ 'mean_test_score' ]\n",
    "test_stds = gsearch2_1.cv_results_[ 'std_test_score' ]\n",
    "train_means = gsearch2_1.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = gsearch2_1.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "pd.DataFrame(gsearch2_1.cv_results_).to_csv('my_preds_maxdepth_min_child_weights_1.csv')\n",
    "\n",
    "# plot results\n",
    "test_scores = np.array(test_means).reshape(len(max_depth), len(min_child_weight))\n",
    "train_scores = np.array(train_means).reshape(len(max_depth), len(min_child_weight))\n",
    "\n",
    "for i, value in enumerate(max_depth):\n",
    "    pyplot.plot(min_child_weight, -test_scores[i], label= 'test_max_depth:'   + str(value))\n",
    "#for i, value in enumerate(min_child_weight):\n",
    "#    pyplot.plot(max_depth, train_scores[i], label= 'train_min_child_weight:'   + str(value))\n",
    "    \n",
    "pyplot.legend()\n",
    "pyplot.xlabel( 'max_depth' )                                                                                                      \n",
    "pyplot.ylabel( 'Log Loss' )\n",
    "pyplot.savefig('max_depth_vs_min_child_weght_1.png' )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 经过图像分析之后树的深度取7 （max_dpath=7）,节点权重取5（min_child_weight': 5）为最佳"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3、再次调整弱分类器数目"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#调整max_depth和min_child_weight之后再次调整n_estimators(6,4)\n",
    "xgb2_3 = XGBClassifier(\n",
    "        learning_rate =0.1,\n",
    "        n_estimators=250,  #数值大没关系，cv会自动返回合适的n_estimators\n",
    "        max_depth=7, #已结确定的最优值\n",
    "        min_child_weight=5, #已结确定的最优值\n",
    "        gamma=0,\n",
    "        subsample=0.8, \n",
    "        colsample_bytree=0.8,\n",
    "        colsample_bylevel=0.7,\n",
    "        objective= 'multi:softprob',\n",
    "        seed=3)\n",
    "\n",
    "modelfit(xgb2_3, X_train, y_train, cv_folds = kfold)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEXCAYAAABCjVgAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl8XGXZ//HPlZksTZN0TUv3li60\nBbFA2RGqgBSUxeVBUFDQR9SfiKLogz6+lB+/B8UdFURRARWBB0EREUVlFQFpy06h0H1vk7RJs7RZ\nZq7fH/dJOk2ztWRyksz3/Xqd5uznmjPTuebc9zn3be6OiIgIQF7cAYiISP+hpCAiIm2UFEREpI2S\ngoiItFFSEBGRNkoKIiLSRklBJIOZfcXMfhF3HCJxUVIYYMysxMxWm9kHM+aVmtlaM3t/xrz5Zna/\nmW03s2ozW2pm15jZiGj5RWaWMrO6aFhpZp/KcuwLzGx9No+xLzqKx92/4e7/maXjrTazU7Kx72zo\nq/droJ2XwU5JYYBx9zrgEuCHZlYezf42sNjd7wYws+OAR4F/AbPdfTiwEGgB3pqxu6fcvcTdS4D3\nA982s8P65pXIvjCzZNwxSI5wdw0DcABuBe4AFgBVwLiMZU8AP+5m+4uAJ9rNewb4YMb0WcArQDUh\nyczJWDYnmlcdrXNWxrIzgKVALbABuAIYCuwE0kBdNIzv5HXdAPw52v7fwPQenI/ZwN+BbcAy4Nz9\niQe4Crgt2m4q4MDFwDpgO/BJ4Ejgxei1X59xnOnAw9H7UQn8FhgeLftNdKyd0bG+1INzvBr4r+hY\njUAymt4QvZZlwMkdnItjgM1AImPee4AXo/GjgMXADmAL8P1OzukCYH0ny4YBvwYqgDXAV4G8aFkC\n+F50DlYBl0bnMdnJvlYDp3Sy7OPA8uh9va/1MwMY8ANgK1ATnaNDOnu/4/7/OpCG2APQsJ9vHIwA\nNkX/8S7OmD8USAELutn+IjKSQvRFVw3MiqZnAfXAqUA+8KXoP2dBNL0c+Eo0/Y7oP+BB0babgLdl\nxHl4NN7pl0xGHLdGXwBHRV+CvwXu7GaboYQv7YujbQ6PzsvB+xoPHSeFnwJFwDuBXcC9wBhgQvSl\ndFK0/ozofBUC5cDjwHUZ+97jy6+rc5yx/vPAJGAIcFD0OsdnxNdhwgRWAKdmTP8OuDIafwq4MBov\nAY7pZB+dvl+EhPBHoDSK43XgY9GyTxK+lCdG5/sf7EdSiD5XldH7WQj8GHg8WnYasAQYTkgQc4h+\nGHX2fmvo2aDiowHK3bcTfmEWA7/PWDSCUCy4uXWGmX07qleoN7OvZqx7TDS/jnCV8BvgjWjZB4A/\nu/vf3b0Z+C7hi+k4wi/REuBad29y94eB+4Hzo22bgblmVubu29392X18eb9392fcvYWQFOZ1s/67\ngdXufou7t0THu4dQJNYb8fw/d9/l7n8jfInf4e5b3X0D8E/gMAB3Xx6dr0Z3rwC+D5zUxX67Oset\nfuTu69x9JyHZF0avJd/dV7v7ik72fQfR+2FmpYRfz3dknI8ZZjba3evc/el9ORlmlohi/7K717r7\nasKVwYXRKucCP3T39dHn9Np92X+GDwE3u/uz7t4IfBk41symRq+hlHCFaO7+qrtvynh9b+b9zmlK\nCgOUmV1A+IX2D+BbGYu2E4opxrXOcPcveahX+APhl3Srp919uIc6hQOAg4FvRMvGE4oFWveRJvxK\nnRAtWxfNa7UmWgbwPsKX0Boze8zMjt3Hl7c5Y7yBkIC6MgU4Okpw1WZWTfhCOaCX4tmSMb6zg+kS\nADMbY2Z3mtkGM9sB3AaM7mK/XZ3jVusyli8HPke4mtkaHWt8J/u+HXivmRUC7wWedffWY32McJXy\nmpktMrN3dxFjR0YTrhDXZMzLfP/HZ8bdbnxftD8/dYSiuQnRD5HrCUWNW8zsJjMri1Z9s+93TlNS\nGIDMbAyhPPXjwCeAc83sRAB3ryeUw793X/bp7lsIv67PjGZtJHzZth7TCMUYG6Jlk8ws8/MzOVqG\nuy9y97MJRSz3Ane1HmZfYtoH64DHogTXOpS4+6f6OJ5vRvs81N3LgAsIRRut2h+vq3Pc4Tbufru7\nnxBt5+z5gyBzvaWEL9TTgQ8SkkTrsjfc/XzC+fgWcLeZDe35y6SS8Gt8Ssa8tvefUHwzMWPZpH3Y\nd6b252coMIrdn7MfufsRhB8zs4AvRvM7e7+lB5QUBqbrgXvd/ZHokvlLwM+jX4VE0x81syujBIKZ\nTQSmdbZDMxtFqIx8JZp1F/AuMzvZzPKBLxAqO58kJJ164Etmlm9mCwjJ5E4zKzCzD5nZsKhIZAeh\n2APCL+xRZjasl85Dq/uBWWZ2YRRPvpkdaWZz+jieUkIlcrWZTSD6ksqwBTgwY7qrc7wXMzvIzN4R\nvc+7CFcpqY7WjdwOXAacSKhTaN3PBWZWHl2ZVEezO92PmRVlDoQr0buAa6LboacAnydcGbW+rs+a\n2QQzG06oHO9OfrvjJKP4LzazedFr/gbwb3dfHb2/R0fnrT46H6lu3m/pibgrNTTs2wCcQ/gFNbzd\n/IeAazKmjwYeIPynrwZeBq4BRkXLLyL8Z2m982Yrocx5TMY+3kOoMKwBHiOquI2WHRzNq4nWeU80\nvwD4K6EYawewCDghY7ubCUUA1XR+99H/ZEwvoJvK6Wi9gwh3LFVE+3+YUBexT/HQcUVzMmP99WRU\n4hO+CL+acU6WROfzecKX/PqMdc8G1kbHuqIH53g1e1ZMH0qo+6klVMbf39E5zFh/MuEL/M/t5t8W\nvd91hB8B53Sy/YLo9bcfZhDqrm6Lzvc64GvsvvsoSbiSrSLcfXQ54crCOjnO6g6O8T/Rsk8SKs1b\nX+/EaP7JhDuO6th9p1dJd++3hu4Hi06wiEhWmNnpwE/dfUq3K0vsVHwkIr3KzIaY2RlmloyK0b5O\nuMlBBgBdKciAYGZvA/7S0TIPd09JP2FmxYSisNmEeo8/A5919x2xBiY9oqQgIiJtVHwkIiJtBlwj\nW6NHj/apU6fGHYaIyICyZMmSSncv7269AZcUpk6dyuLFi+MOQ0RkQDGzNd2vpeIjERHJoKQgIiJt\nlBRERKSNkoKIiLRRUhARkTZKCiIi0kZJQURE2uRMUti6YxdPrqiMOwwRkX4tZ5LC96+/jppbz6Nh\nV2PcoYiI9Fs5kxQ+MBNOTyxiW+WW7lcWEclROZMUCoaNBaCmcmPMkYiI9F85kxSKR4wDoH7bppgj\nERHpv3ImKZSODkmhsUbFRyIincmZpDBs9HgAUrVbY45ERKT/ypmkkD90FC3kQb2SgohIZ3ImKZCX\nR40NI9lQFXckIiL9Vu4kBaA2MYLCpm1xhyEi0m9lLSmY2c1mttXMXu5kuZnZj8xsuZm9aGaHZyuW\nVjsLRjK0WUlBRKQz2bxSuBVY2MXy04GZ0XAJcGMWYwGguWgkpanqbB9GRGTAylpScPfHga5+lp8N\n/NqDp4HhZjYuW/EApIvLGUUNu5pT2TyMiMiAFWedwgRgXcb0+mjeXszsEjNbbGaLKyoq9vuAeSVj\nKLZGKrdt3+99iIgMZnEmBetgnne0orvf5O7z3X1+eXn5fh8wWRaauqhWUxciIh2KMymsByZlTE8E\nsvptPWRESAr1VUoKIiIdiTMp3Ad8OLoL6Rigxt2z2jBRychQZbGrWk1diIh0JJmtHZvZHcACYLSZ\nrQe+DuQDuPtPgQeAM4DlQANwcbZiaTVsdKiyeOrF1zjprGwfTURk4MlaUnD387tZ7sCns3X8jrQ2\nn3346Oa+PKyIyICRU080kyyk1kpIqv0jEZEO5VZSAGqSoylqVFIQEelIziWFhsJySpsq4w5DRKRf\nyrmk0Fw8hpG+jeZUOu5QRET6nZxLCpSOYwzVbN2xM+5IRET6nZxLCvnDDiDfUlRtVV/NIiLt5VxS\nKBo5EYAdW9d1s6aISO7JuaQwbExoWaNh2/qYIxER6X9yLimUlYek0Fyt9o9ERNrLuaRgpQeEv7Wb\nY45ERKT/ybmkQLKQHVZGfoMaxRMRaS/3kgJQmz+a4sb976xHRGSwysmksLa5jJLmSkKbfCIi0ion\nk8KoA6Yw1razvUGtpYqIZMrJpJA3bDzlVLOhqjbuUERE+pWcTApDyqeQMKdq8+q4QxER6VdyMimU\njZ0GQO2W1fEGIiLSz+RkUigZMxWApqq18QYiItLP5GRSsGGh/SN2qKkLEZFMOZkUKCyhLq+Uwno1\ndSEikik3kwKwo/AAyhrV1IWISKacTQqNQ8dTnq6gdpeeVRARaZXVpGBmC81smZktN7MrO1g+xcwe\nMrMXzexRM5uYzXj2UDaRCVbFhmr1wCYi0iprScHMEsANwOnAXOB8M5vbbrXvAr9290OBq4FvZiue\n9vJHTabMGti8ZWtfHVJEpN/L5pXCUcByd1/p7k3AncDZ7daZCzwUjT/SwfKsKY1uS63evKqvDiki\n0u9lMylMADL7vFwfzcv0AvC+aPw9QKmZjcpiTG1aH2Cr37q6Lw4nIjIgZDMpWAfz2jdLegVwkpk9\nB5wEbABa9tqR2SVmttjMFldU9E6T1zY89MCWWvl4r+xPRGQwyGZSWA9MypieCOzxYIC7b3T397r7\nYcB/R/Nq2u/I3W9y9/nuPr+8vLx3ois5gGbLpzDRUe4SEclN2UwKi4CZZjbNzAqA84D7Mlcws9Fm\n1hrDl4GbsxjPnvLyqCmawIimDTSn0n12WBGR/ixrScHdW4BLgQeBV4G73P0VM7vazM6KVlsALDOz\n14GxwDXZiqcjTaVTmMQWNmzXbakiIgDJbO7c3R8AHmg372sZ43cDd2czhq7kjTqQyVue5pnKOqaO\nHhpXGCIi/UbOPtEMMPSAGQy1RrZuVGupIiKQ40mhZNxMAOo2L485EhGR/iGnk4KNPBCAVOXKmCMR\nEekfcjopMHwyaYymyhVxRyIi0i/kdlJIFlJXOJaJvpldzam4oxERiV1uJwXCbamTbQsrK+rjDkVE\nJHY5nxQS5TOYaptZXlEXdygiIrHL+aRQMmE2I62O9RvUX7OISM4nhfwxBwHQsPG1mCMREYlfzicF\nRs0AwKpejzkQEZH4KSkMn0LKkpTWraZFDeOJSI5TUkgkqRs6halsZO22hrijERGJlZIC4KNmcKBt\n4hO/WRJ3KCIisVJSAIaOn8Nk28K7D+6lDnxERAYoJQUgf8wsCixFxbplcYciIhIrJQWA0bMAaFnz\ndMyBiIjES0kBoDw8qzAyvZ0du5pjDkZEJD5KCgBFZewsnsBBeet4bVNt3NGIiMRGSSFiY+cwy9bx\n6qYdcYciIhIbJYVI4fhDmJG3kdc2bIs7FBGR2CgpRGzMXPJJsX39q3GHIiISGyWFVmPnApBf9Zo6\n3BGRnKWk0GrUTNKWYCZreXlDTdzRiIjEIqtJwcwWmtkyM1tuZld2sHyymT1iZs+Z2YtmdkY24+lS\nfhHpEQcy29bx/Lrq2MIQEYlT1pKCmSWAG4DTgbnA+WY2t91qXwXucvfDgPOAn2Qrnp5Ijj+UQxNr\neU5JQURyVDavFI4Clrv7SndvAu4Ezm63jgNl0fgwYGMW4+neuHkcQAVr1q6JNQwRkbhkMylMANZl\nTK+P5mW6CrjAzNYDDwCf6WhHZnaJmS02s8UVFRXZiDUYfxgAo3a8SkVtY/aOIyLST2UzKVgH87zd\n9PnAre4+ETgD+I2Z7RWTu9/k7vPdfX55eRZbMh13KACH2CqWrNmeveOIiPRT2UwK64FJGdMT2bt4\n6GPAXQDu/hRQBIzOYkxdKxpGeuQM5iVWsWi1HmITkdyTzaSwCJhpZtPMrIBQkXxfu3XWAicDmNkc\nQlLIYvlQ9/LGz+Ow5CqeWaWkICK5p9ukYGbTzawwGl9gZpeZ2fDutnP3FuBS4EHgVcJdRq+Y2dVm\ndla02heAj5vZC8AdwEXu3r6IqW+NP4zR6Uq2bFxLXWNLrKGIiPS1ZA/WuQeYb2YzgF8Sfu3fTqgD\n6JK7P0CoQM6c97WM8aXA8fsScNZNOByAt9hynl2znRNnqTc2EckdPSk+Ske/+t8DXOfulwPjshtW\njMbNwy3BEYkVKkISkZzTk6TQbGbnAx8B7o/m5WcvpJgVFGMHHMIJRat4ckVl3NGIiPSpniSFi4Fj\ngWvcfZWZTQNuy25YMZt4FLNTb/DSevXEJiK5pduk4O5L3f0yd7/DzEYApe5+bR/EFp+JR1KQbmC6\nr+OpFVVxRyMi0md6cvfRo2ZWZmYjgReAW8zs+9kPLUaTjgTg6PwVPPGGipBEJHf0pPhomLvvAN4L\n3OLuRwCnZDesmI2YBnn5nJpYwj/fiPWxCRGRPtWTpJA0s3HAueyuaB7czGDWaRzIBlZXNbCmqj7u\niERE+kRPksLVhAfQVrj7IjM7EHgju2H1A1NPYLxvYRxV/OPVrXFHIyLSJ3pS0fw7dz/U3T8VTa90\n9/dlP7SYTQnP1J0zYhV/X7o55mBERPpGTyqaJ5rZH8xsq5ltMbN7zGxiXwQXq7EHQ9EwTi9dzqLV\n26luaIo7IhGRrOtJ8dEthKYtxhP6Q/hTNG9wy0vAlOM5aNeLpNLOI8tUhCQig19PkkK5u9/i7i3R\ncCuQGw0CbX2Vwh2rmZTYxtV/Whp3NCIiWdeTpFBpZheYWSIaLgBy44mu824H4IszNlDfmNLTzSIy\n6PUkKXyUcDvqZmAT8H5C0xeD35g5UDqet9kLNKXS/O2VLXFHJCKSVT25+2itu5/l7uXuPsbdzyE8\nyDb4mcGMdzB80xNMHlbA/S+27zhORGRw2d+e1z7fq1H0ZzNOwRp3MCe9jEeXVVBR2xh3RCIiWbO/\nScF6NYr+7MAFAHyzINxw9ftn18cXi4hIlu1vUoi3y8y+NGQETDmBkUMLmD9lBP+7aB1x9xgqIpIt\nnSYFM6s1sx0dDLWEZxZyx+wzYOtSPjrXWVlZz6LV2+OOSEQkKzpNCu5e6u5lHQyl7t6Tvp0Hj4NC\nd9Sn5C2htCjJr55aHWs4IiLZsr/FR7ll5DQYewgFb/yF84+azF9f3syG6p1xRyUi0uuUFHpqVw2s\nfZKL3zoEgF8/uTreeEREsiCrScHMFprZMjNbbmZXdrD8B2b2fDS8bmbV2YznTfnQ3QCMW/8gw4bk\n84t/rqJmp55wFpHBJWtJwcwSwA3A6cBc4Hwzm5u5jrtf7u7z3H0e8GPg99mK500bMxvGHgIv3c2v\nP3oUKXd+89TquKMSEelVPWk6u6O7kNZFzWkf2MWmRwHLo/4XmoA7gbO7WP984I59C7+PHfI+WP8M\nhxRX8/aDyvnlE6toaGqJOyoRkV7TkyuF7wNfJDSbPRG4Avg54Uv+5i62mwCsy5heH83bi5lNAaYB\nD/cgnvgcErXu8aszufQdM9ne0Mw7f/B4vDGJiPSiniSFhe7+M3evdfcd7n4TcIa7/y8woovtOnrq\nubOnvs4D7nb3VIc7MrvEzBab2eKKiooehJwlI6bCtBMB54hJZZw6dyzVDc1U1qnpCxEZHHqSFNJm\ndq6Z5UXDuRnLunq0dz0wKWN6ItBZi3Ln0UXRkbvf5O7z3X1+eXnMXTnM/yhUr4UVD3Pl6bPZ2Zzi\nB39/Pd6YRER6SU+SwoeAC4Gt0XAhcIGZDQEu7WK7RcBMM5tmZgWEL/772q9kZgcRrjie2sfY43HQ\nu2DoGFh8M9PLS7jwmCnc/sxanl2rp5xFZODrSdPZK939THcfHQ1nuvtyd9/p7k90sV0LIWk8CLwK\n3OXur5jZ1WZ2Vsaq5wN3+kBpUChZAIkCWPYA1KznitMO4oCyIq6850WaWtJxRyci8qb05O6jidGd\nRlvNbIuZ3WNmE3uyc3d/wN1nuft0d78mmvc1d78vY52r3H2vZxj6tYsfAAye/Q0lhUlKi5K8vqWO\nnz62Iu7IRETelJ4UH91CKPYZT7h76E/RvNw1YgrMOAWe/RWkWvjb5Sfx7kPHcf3Dy1m+tS7u6ERE\n9ltPkkK5u9/i7i3RcCsQc21vPzD/o1C7CW48HoCvn3kwKXfOvuEJmlMqRhKRgaknSaHSzC4ws0Q0\nXABUZTuwfm/WaTDyQMgvBHfKSwu57gPzqG9M8eOH3og7OhGR/dKTpPBR4FxgM7AJeD9wcTaDGhDy\nEnDC5bDpBVjxEABnvnU8o0sK+NHDy3nkta0xBygisu96cvfRWnc/y93L3X2Mu58DvLcPYuv/Dj0v\n3In0u4sgunnqsS++nUMmlPHp25/l5Q018cYnIrKP9rdBvM/3ahQDVbIAFl4LjbXw+l8BGFqY5OaP\nHElLynnPT/6lfhdEZEDZ36TQURMWuenwD8PI6fCPqyAdWukYU1bE/ZedQFEywcW3PKNmMERkwNjf\npDAwHjTrC4l8yMuHitfghd0tdcwaW8rPPnwEa7c1cO7PnmKjrhhEZADoNCl00mT2DjOrJTyzIK0+\n/TRMOAIe+QY07/7yP276aH7zsaNZU9nAgu88yqrK+hiDFBHpXqdJwd1L3b2sg6HU3ZN9GWS/Zwan\n/F/YsQGeuWmPRUdOHckfLz2etDunfv8xnlxRGVOQIiLdUx/NvWXa22DmO+Gx70Dt5j0WHTJhGA9e\nfiJTRhVz4S+f4abHVzBQmnoSkdyipNCbFl4LqUb4+9f2WjS9vIQ/XnoC75w7lm888BqfvG2JKqBF\npN9RUuhNo6aHZrVf/F944x97LS4pTPKTDx3Of58xh0deq+CYbzzEyd97VFcNItJvKCn0ts8shvI5\ncO+noH7v+gMz4+MnHsifLzuBQyYMY0VFPfOu/jvLNtfGEKyIyJ6UFHpb/hB43y+gvgKuP7LtSef2\nZo4t5Z5PHcdX3zWHusYWTrvucf7r7hfZXLOrjwMWEdlNSSEbDjgETrsGdm6DJZ23Mp7IM/7zbQey\n+L9P4WMnTOOuxes47tqH+NZfX1N9g4jEwgZaefb8+fN98eLFcYfRvXQavj0NGnfAJ5+AsQd3u8m6\nbQ2cc8O/qKpvwgzKSwq5/eNHM2NMaR8ELCKDmZktcff53a6npJBFtVvgZ2+DghK45FEoKuvRZsu3\n1vHLJ1Zx56K1uMPhk4fzviMm8u5DxzNsSH5WQxaRwUlJob9Y8yTccjoUjYAvvhGaxeihyrpG7lmy\nnuv+8QY7m1OYwRlvGcfCgw/gxFnlShAi0mNKCv3JklvhT5+Fwy6As64PT0DvA3fnpQ013LNkPb/9\n91pa0uE9O3raSE6eM4Z3zB7D9PISbB/3KyK5Q0mhv/nBIVCzDhZ8GRZcud+7SaWd59Zu57I7nmN7\nQzM7m0PLrJNGDuHk2WN5++wxHD1tJEX5id6KXEQGASWF/sYd7v0/8MLtMOJA+OxzvbLbDdU7eeS1\nrTz82lYeWbYV93AhcuSUkRwxdQTzp4zg8MkjGDG0oFeOJyIDk5JCf5Rqhrs+DMseCE1iHPOpXt39\nruYUT62o4p9vVLJk7XZeXFfd1sZ5YTKPk+eM4eDxwzh4fBkHHVDK2NIi8vJU5CSSC/pFUjCzhcAP\ngQTwC3e/toN1zgWuIvTR8IK7f7CrfQ7opADQ0gTfnw0NVfDOa+C4S7N2qF3NKV5YV82za6t5eUMN\nL2+sYU1VQ9tyM5hRXsKUUcVMGTV099+RxUwYMYT8hB5jERksYk8KZpYAXgdOBdYDi4Dz3X1pxjoz\ngbuAd7j7djMb4+5d9ng/4JMChCuGe/4Tlt4LZRPhcy9BXt98AdfsbGbpxh2sqKhjTVU9a6oa+Ocb\nlTS2pEi3+ygUJvM4atpIpowqZuqooUweWczU0eGv6ixEBpaeJoVs9otwFLDc3VdGAd0JnA0szVjn\n48AN7r4doLuEMGgk8uF9v4Qhw8OdSXdfBOf8FAqKs37oYUPyOXb6KI6dPmqP+e7O1tpG1lQ1sLqq\nnrVVDdz+zFqeXbOdJ1dUkWqXMQwozM/jrROHM7qkkFElBYwcWsCokkJGR39HlRQwamgBw4bk684o\nkQEim0lhArAuY3o9cHS7dWYBmNm/CEVMV7n7X9vvyMwuAS4BmDx5claC7XOJJLz7Ohg1E/72Vahe\nBx+4DYZNiCUcM2NsWRFjy4o4atpIAK447aC25dUNTXskjC21u6iqa+KJNyqpb2ohz6ztVtm99t12\njNBSbDKRRzLPOHf+JIYX5zNsSD7DiwsYXpzPiOJ8yoaEeYVJXY2I9LVsJoWOfhq2/9ZIAjOBBcBE\n4J9mdoi7V++xkftNwE0Qio96P9SYmIU6hZHT4PeXwE0nwftvhmknxh3ZXsKXdgFvnTS803WaU2m2\n1zdRWdfEtvomquobqaxroqqukaq6Jh5cupmWVJqGphYam9Nc/8jybo9rQFF+gmSesbM5xfDifPLM\nSOQZHzhyEkMLkgwpSFDcNiQZWpigpDCfkqIkxfkJCvPzKEwmSKhSXaRb2UwK64FJGdMTgY0drPO0\nuzcDq8xsGSFJLMpiXP3P7HfBxx+Gn50EvzozdO153GV9Vs/QW/ITeYwpK2JMWVGHy7/FoXtMp9NO\nXVMLNQ3NVDc0s72hieqdzdQ0NFGzs5nbnl5LVX0jRfl5tKSdtDs7drWQTjstaee6f7yxzzHmGeSZ\nkUo7Bck88sxobEkxtCCJGdQ3pTBgeHEo8sozOGfeBAqTeRTmJ8LfZEgyIdlkzg9/i/J3jxcmExQk\n8yhI5ikpyYCQzYrmJKGi+WRgA+GL/oPu/krGOgsJlc8fMbPRwHPAPHev6my/g6KiuTONtfDDedBQ\nCVNOgHNugBFT446q30qlnZ3NKRqaWmhoTNHQlGJncwt1jSnqdrVQu6uZXc0pGlvSNLak28b/9MJG\ntjU0Mawovy3RDC1IkHZoaGoBQoJLu9OccvKMvSrh91eehaK6dNrJT+RhFq6wAIYUJMgzo6EpRUlh\nEgPqGlsoK0piZtTsbGZEcT6YUd3QBMCooYWYQVVdI+WlRZiFq6uLjp9GQcLIT+SRTOSRyAvJ0MxI\nRMkuL8/Iazfeusz2mN86HY1jWJRc86L9tl8nHKt1Wbt9RvMsY9vM9Y3d06qL6j2x330UBXEGcB2h\nvuBmd7/GzK4GFrv7fRbe8e8BC4EUcI2739nVPgd1UoDwkNtzt4VmMXA447sw/6P73DSG9C53pykV\nkktjc5rGltTe4y1pGjOSUGMKTl1+AAASjElEQVRLisbmNLtaUjS3OP+7aC0OpN3ZVt/EsCH5uIc7\nwhwoKYwSU2MLRQUJ3MNtxQXJPHBoTKXJz8vDcVpSjhOSjPve5bKDjUX/uO9OrKm0k4yuvlJpJ5kI\n4y2pcDaSCcMwmlPpcHt1lIALolutm1JpCqP5TS3pcJ4J40BbnVZjS6rtbrtdzWHcgJ3NKYYUhPk7\nm0LLAsUFyWi6JYxbeD+LoyRf35RiaLRNfVOKksIEYNQ1hh8jpYVh+9rGFkqLwja1u8I4wJjSQh68\n/KT9O4f9ISlkw6BPCq2q18FPT4Bd1TDtJDj7ehg+SCrZJSvSaac5nSaVdppbwnhTS5qWVCh62z2E\nL9G0O54xnnYnlQ4JMO27/7Yu87bxztfZvR7ReObyaP10GL/1ydVAWH9LbSNjSgsB2FrbSHlJIeBU\n1IZ+RUaVhGWVdY2Mip7Or6pvYmRxAQ5sb2hiRDTeehU1PGowcvvO5rbx6obmtoYka3aGcQd27Gym\nbEg+RFeOQNsXce2uFkoyxjO/uEsKk4BT1xiSQuYXfnE03tDY0pYsGppaGLLH+J5JpX3yaR1vTVAT\nhhfxjy8s2K/Ph5LCYOAeOun58xVh+vRvwREXhzuXRET2QU+TwsCqycw1ZqHo6LLnoLAEHrgCfnI0\nLL2v024+RUTeDCWFgWDEFPivNXDeHbBjA9x1IXxzEqz+V9yRicggo6QwUJjB7DPgyvWhT4ZUI9x6\nBtz+AdiytPvtRUR6QElhoEkk4fALw5XDKVfBG3+HG4+FP3wKqlbEHZ2IDHBKCgNVQTGccDl8cTmU\nTYAX74QfHw7fng5bXul+exGRDigpDHTFI+HzS+Hzr4Xk0FAJNx4Ht58H63PkLi0R6TW6t3GwKB0b\nkkPDNnjm5/D4t+H1v0BhGZx5Hcx+NyQL445SRPo5PacwWDXWwY3HQ+2mUCmNQdl4+PB9MHpG3NGJ\nSB/rD/0pSJwKS+BzL0A6DSsfgd9/HHZshOuPgMJhsPAbMOt0GDqq+32JSM7QlUIuqd0Cz/8WHvsW\ntOwCDOaeDfM+BAeepOIlkUFMzVxI59xh0/Pw0t3w759CugXyknD8Z2H+x2Lr6EdEskdJQXqmpRFW\nPgbP/Rpe/VOYV1gWbned/S4oP6jr7UVkQFBSkH23bVXo5KehCpobwrz8Yjj+c3DwOUoQIgOYkoK8\nOTs2wi3vCs89NO4I85JD4MiPwcx3wuRjIVkQb4wi0mNKCtJ7dmwKRUuPXAO7amjr0qV4VGhqY8ap\nUDYuxgBFpDtKCpIdjXWw6nG4/3LYuQ1SoUMTCobC0Z8KVxET50NeIt44RWQPSgqSfe6wdSnc8cGQ\nIFqLmfKSUDQChoyAj/4Fho6ON04R0cNr0gfMYOzB4SE5gJ3bYcUjoeXWl34HDRXwnemQPyQ8MHfa\nNaEuQre8ivRbulKQ7EinYeNzsPpx+NcPQ8JolSwKt72e8nWYchyMmBYSjIhkjYqPpH9Jp2DLy6G3\nuMe/G4qbWiusEwUw58yQIKYcD6MPgjw14CvSm5QUpH9zh4plsOZf8Mg3obFmd6V1XjJcSRQNg/+4\nBca+JXQuJCL7rV8kBTNbCPwQSAC/cPdr2y2/CPgOsCGadb27/6KrfSopDFLusH01rHkyDC/fHbXP\nBFgCpr89XElMPg7GHRrudhKRHos9KZhZAngdOBVYDywCznf3pRnrXATMd/dLe7pfJYUcsmPj7iTx\nwh27n7KG8KT1we+FSUfB5GNg9CzVS4h0oT/cfXQUsNzdV0YB3QmcDaiXeemZsvHwlveH4d3fh/oq\nWPfv0JjfMzeFLkifvy2sm5eEwlIoKA2dCo0/LPRKJyL7JJtJYQKwLmN6PXB0B+u9z8xOJFxVXO7u\n6zpYRyT0/TD7jDC8/SuhyKlqOdxxHuyqDc9J7NwOt703rJ8sCknibZeHW2fHHqJnJkS6kc2k0NG1\nfPuyqj8Bd7h7o5l9EvgV8I69dmR2CXAJwOTJk3s7ThmozGD0TPjMkt3zdtWEW2E3PAtPXh/abnrw\nK7uX5+XD1BNCsdPUE2DiUZBf1Pexi/RT2axTOBa4yt1Pi6a/DODu3+xk/QSwzd2HdbVf1SnIPqvb\nClteCU9fb3kFlt4LTfXRQoOCYphzdnQ1MTdcUZSMiTVkkd7WH+oUFgEzzWwa4e6i84APZq5gZuPc\nfVM0eRbwahbjkVxVMiYM098eps/5SbiiWPNUuCV2y8uw4iF44fbd2+Tlh/XHHw4TjoAJh6voSXJC\n1pKCu7eY2aXAg4RbUm9291fM7GpgsbvfB1xmZmcBLcA24KJsxSOyh6JhcNDCMLSqr9x9RbH5JXjl\nD/DG33YvTxTCrNNgzFwYMxvK58Co6ZDI7/v4RbJED6+JdKWxFja9ABuWwL9+BE11u5+fAMBC204z\n3wlj5kD57PB35IFKFtKvxP6cQrYoKUjsmndC5euw9TWoeBWW/Cpq2ynz/1KULPKL4ehPhGRRPjtK\nFno6W/qekoJIX2tqCMmi4jXY+io8+yvYWU3nyeKTu4uhRkxVspCsUlIQ6S+a6ve8snj21x0kC0Ki\nmLUw1FmMnRvuhho2WY0DSq9QUhDp7xrroHIZVLwekkXFMlj5CLQ07rleYSkc8v6ogntO+Dt0VDwx\ny4DVH25JFZGuFJZEt7sesef8xtpwVbH1FXj02nCl8dxvIN2SsZJBURm89YNRoogquYvK+vQlyOCj\nKwWRgcAdajeH22W3vhquLF65NzQS6Ond6yUKw8N4h39495XF6FmhHkNymoqPRHJBOg01a0Oi2LoU\nnr4RGqpCEsmss0gUhI6OhpbDcZ+BUTPCMGKKbp3NEUoKIrks1QLbVoZEUfl6aFW2YVtoL2qPYihC\nw4HpFhg6Bk68IjyQN2oGlI5XJfcgoqQgIh1r2AZVK+DeT0HzLmjZGXWPyp5FUZYXEsbMU3dfWbQO\napZ8wFFFs4h0rHhkGD7T7sdVOg21m0Jz5NtWwGPfgfqt8PrfoOU+9iiOykuGIqrikXDUJ2DkNBg+\nOQwlY9Xh0QCmKwUR6V6qGarXhoRRtTw0+VFfEZJDqnHv9ZNDIFkYKriPuywUSY08EIZPgWRB38cv\nKj4SkT7S1BASRvVaqF4DT/wgtA/VsiujifIMiShZJItCpffIA8MwYqr6tsgiJQURiZ97qMPYtiJU\nfD/8P7vrMZrq2eup7kRhVOldDsf+n4yEMS3caiv7TUlBRPq/hm2wbVVIGNtWwqKfR3dJ5UG6ec91\nE/mh3qN4ZLjKSBbBmT8Mve8NLVc9RjeUFERkYNtVExLG7y/ZXRxVX9l5PUZBSajLaKwJdRdnXx+u\nMpQwACUFERnMUs1Qsy5cXVQuhyeug5aGUDTVUcLIHxrqK5JFcOIXw91SIw+EsgmQl+j7+GOgpCAi\nuamlKVR6b1sJf/kv2LEeCobursvYg0UP7zWHK4rjPxcSxohp4WnvZGEsLyEblBRERNpLp2DHxpAw\n7v98VCy1M+okycBTe67fVvE9OiSPRCGc/WMYNglKDxhQVxlKCiIi+8I91FlsXwX3fjoki5ZdnVd8\nQ1ThXRiSxZH/GT3ANynUaZRN6FfNhCgpiIj0pqYGqFkfGiCsXguPfy/0fZHaFfrGaH97bfuiqWMv\nDc9itA6FJX0avpKCiEhfat4FOzaEB/i2rwlXHM/d1nnRFBbumJp1WlSPMXX3UDqu14um1PaRiEhf\nyi+KWpidvnveqVfvHt+5Pdxiu331nsNrf9q7tz0It9dOPX7PZDH+cBg2IYsvQklBRKRvDBkBE0bA\nhMP3XpZqDkVT7RPG8n+EodXI6XDZs1kNM6tJwcwWAj8EEsAv3P3aTtZ7P/A74Eh3V9mQiOSWRH70\n7MS0jpfv3B6KpErHZT2UrCUFM0sANwCnAuuBRWZ2n7svbbdeKXAZ8O9sxSIiMqANGRGGPpDN+6WO\nApa7+0p3bwLuBM7uYL3/B3wb2JXFWEREpAeymRQmAOsyptdH89qY2WHAJHe/v6sdmdklZrbYzBZX\nVFT0fqQiIgJkNyl01AJV2/2vZpYH/AD4Qnc7cveb3H2+u88vLy/vxRBFRCRTNpPCemBSxvREYGPG\ndClwCPComa0GjgHuM7Nu76MVEZHsyGZSWATMNLNpZlYAnAfc17rQ3WvcfbS7T3X3qcDTwFm6+0hE\nJD5ZSwru3gJcCjwIvArc5e6vmNnVZnZWto4rIiL7L6vPKbj7A8AD7eZ9rZN1F2QzFhER6V7/acJP\nRERiN+AaxDOzCmDNfm4+GqjsxXAGI52jrun8dE/nqGtxnZ8p7t7t7ZsDLim8GWa2uCetBOYynaOu\n6fx0T+eoa/39/Kj4SERE2igpiIhIm1xLCjfFHcAAoHPUNZ2f7ukcda1fn5+cqlMQEZGu5dqVgoiI\ndEFJQURE2uRMUjCzhWa2zMyWm9mVccfTH5jZajN7ycyeN7PF0byRZvZ3M3sj+ts3PXv0E2Z2s5lt\nNbOXM+Z1eE4s+FH0mXrRzDroZ3Fw6eT8XGVmG6LP0fNmdkbGsi9H52eZmZ0WT9R9x8wmmdkjZvaq\nmb1iZp+N5g+Yz1BOJIWMXuBOB+YC55vZ3Hij6jfe7u7zMu6bvhJ4yN1nAg9F07nkVmBhu3mdnZPT\ngZnRcAlwYx/FGKdb2fv8APwg+hzNi5q3Ifo/dh5wcLTNT6L/i4NZC/AFd59DaPn509F5GDCfoZxI\nCvS8FzgJ5+VX0fivgHNijKXPufvjwLZ2szs7J2cDv/bgaWC4mWW/E90YdXJ+OnM2cKe7N7r7KmA5\n4f/ioOXum9z92Wi8ltAY6AQG0GcoV5JCt73A5SgH/mZmS8zskmjeWHffBOEDDoyJLbr+o7Nzos/V\nbpdGxR83ZxQ55vT5MbOpwGGE/ucHzGcoV5JCl73A5bDj3f1wwiXsp83sxLgDGmD0uQpuBKYD84BN\nwPei+Tl7fsysBLgH+Jy77+hq1Q7mxXqOciUpdNcLXE5y943R363AHwiX9ltaL1+jv1vji7Df6Oyc\n6HMFuPsWd0+5exr4ObuLiHLy/JhZPiEh/Nbdfx/NHjCfoVxJCl32ApeLzGyomZW2jgPvBF4mnJeP\nRKt9BPhjPBH2K52dk/uAD0d3kBwD1LQWEeSSdmXg7yF8jiCcn/PMrNDMphEqU5/p6/j6kpkZ8Evg\nVXf/fsaiAfMZymonO/2Fu7eYWWsvcAngZnd/Jeaw4jYW+EP4DJMEbnf3v5rZIuAuM/sYsBb4jxhj\n7HNmdgewABhtZuuBrwPX0vE5eQA4g1CB2gBc3OcB97FOzs8CM5tHKPZYDXwCIOpp8S5gKeGunE+7\neyqOuPvQ8cCFwEtm9nw07ysMoM+QmrkQEZE2uVJ8JCIiPaCkICIibZQURESkjZKCiIi0UVIQEZE2\nSgoiItJGSUGkB8xsXrsmoc/qrSbYzexzZlbcG/sSebP0nIJID5jZRcB8d780C/teHe27ch+2SeTA\ng2ASA10pyKBiZlOjDk5+HnVy8jczG9LJutPN7K9RK7H/NLPZ0fz/MLOXzewFM3s8ahrlauADUScy\nHzCzi8zs+mj9W83sxqhzlZVmdlLUWuirZnZrxvFuNLPFUVz/N5p3GTAeeMTMHonmnW+h86OXzexb\nGdvXmdnVZvZv4Fgzu9bMlkatk343O2dUco67a9AwaAZgKqFJhXnR9F3ABZ2s+xAwMxo/Gng4Gn8J\nmBCND4/+XgRcn7Ft2zSh45k7CS1eng3sAN5C+NG1JCOWkdHfBPAocGg0vRoYHY2PJzSDUE5ofuRh\n4JxomQPntu4LWMbuq/3hcZ97DYNj0JWCDEar3L213ZklhESxh6hp4+OA30Vt1PwMaG3Y7V/ArWb2\nccIXeE/8yd2dkFC2uPtLHloNfSXj+Oea2bPAc4TeyDrq/e9I4FF3r3D3FuC3QGuT5ilC65sQEs8u\n4Bdm9l5Cuzkib1pONIgnOacxYzwFdFR8lAdUu/u89gvc/ZNmdjTwLuD5qLG3nh4z3e74aSAZtRJ6\nBXCku2+PipWKOthPR+3rt9rlUT2Ch0YejwJOJrT6eynwjh7EKdIlXSlITvLQ8ckqM/sPaOtA/a3R\n+HR3/7e7fw2oJLR3XwuUvolDlgH1QI2ZjSV0bNQqc9//Bk4ys9FRf8bnA4+131l0pTPMQ3/InyN0\ncCPypulKQXLZh4AbzeyrQD6hXuAF4DtmNpPwq/2haN5a4MqoqOmb+3ogd3/BzJ4jFCetJBRRtboJ\n+IuZbXL3t5vZl4FHouM/4O4d9WlRCvzRzIqi9S7f15hEOqJbUkVEpI2Kj0REpI2Kj2TQM7MbCD1i\nZfqhu98SRzwi/ZmKj0REpI2Kj0REpI2SgoiItFFSEBGRNkoKIiLS5v8DtsmMeCGnXIoAAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xcfa8e80>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "cvresult = pd.DataFrame.from_csv('1_nestimators.csv')\n",
    "        \n",
    "# plot\n",
    "test_means = cvresult['test-mlogloss-mean']\n",
    "test_stds = cvresult['test-mlogloss-std'] \n",
    "        \n",
    "train_means = cvresult['train-mlogloss-mean']\n",
    "train_stds = cvresult['train-mlogloss-std'] \n",
    "\n",
    "x_axis = range(0, cvresult.shape[0])\n",
    "        \n",
    "pyplot.errorbar(x_axis, test_means, yerr=test_stds ,label='Test')\n",
    "pyplot.errorbar(x_axis, train_means, yerr=train_stds ,label='Train')\n",
    "pyplot.title(\"XGBoost n_estimators vs Log Loss\")\n",
    "pyplot.xlabel( 'n_estimators' )\n",
    "pyplot.ylabel( 'Log Loss' )\n",
    "pyplot.savefig( 'n_estimators4_1.png' )\n",
    "\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 最终确定我们的数目取200"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 第四步调整subsample 和 colsample_bytree 参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'colsample_bytree': [0.6, 0.7, 0.8, 0.9], 'subsample': [0.6, 0.7, 0.8]}"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "subsample = [i/10.0 for i in range(6,9)]\n",
    "colsample_bytree = [i/10.0 for i in range(6,10)]\n",
    "param_test2_4 = dict(subsample=subsample, colsample_bytree=colsample_bytree)\n",
    "param_test2_4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Anaconda\\envs\\python3\\lib\\site-packages\\sklearn\\model_selection\\_search.py:761: DeprecationWarning: The grid_scores_ attribute was deprecated in version 0.18 in favor of the more elaborate cv_results_ attribute. The grid_scores_ attribute will not be available from 0.20\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "([mean: -0.58297, std: 0.00361, params: {'colsample_bytree': 0.6, 'subsample': 0.6},\n",
       "  mean: -0.58220, std: 0.00382, params: {'colsample_bytree': 0.6, 'subsample': 0.7},\n",
       "  mean: -0.58125, std: 0.00408, params: {'colsample_bytree': 0.6, 'subsample': 0.8},\n",
       "  mean: -0.58261, std: 0.00388, params: {'colsample_bytree': 0.7, 'subsample': 0.6},\n",
       "  mean: -0.58178, std: 0.00414, params: {'colsample_bytree': 0.7, 'subsample': 0.7},\n",
       "  mean: -0.58012, std: 0.00370, params: {'colsample_bytree': 0.7, 'subsample': 0.8},\n",
       "  mean: -0.58196, std: 0.00323, params: {'colsample_bytree': 0.8, 'subsample': 0.6},\n",
       "  mean: -0.58074, std: 0.00360, params: {'colsample_bytree': 0.8, 'subsample': 0.7},\n",
       "  mean: -0.58013, std: 0.00323, params: {'colsample_bytree': 0.8, 'subsample': 0.8},\n",
       "  mean: -0.58240, std: 0.00385, params: {'colsample_bytree': 0.9, 'subsample': 0.6},\n",
       "  mean: -0.58179, std: 0.00374, params: {'colsample_bytree': 0.9, 'subsample': 0.7},\n",
       "  mean: -0.58045, std: 0.00331, params: {'colsample_bytree': 0.9, 'subsample': 0.8}],\n",
       " {'max_depth': 7, 'min_child_weight': 5},\n",
       " -0.58806154462978555)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb2_4 = XGBClassifier(\n",
    "        learning_rate =0.1,\n",
    "        n_estimators=200,  #最优值\n",
    "        max_depth=7,\n",
    "        min_child_weight=5,\n",
    "        gamma=0,\n",
    "        subsample=0.8,\n",
    "        colsample_bytree=0.8,\n",
    "        colsample_bylevel = 0.7,\n",
    "        objective= 'multi:softprob',\n",
    "        seed=3)\n",
    "\n",
    "#采用GridSearchCV来查看最优的树的行列采样\n",
    "gsearch2_4 = GridSearchCV(xgb2_4, param_grid = param_test2_4, scoring='neg_log_loss',n_jobs=2, cv=kfold) \n",
    "gsearch2_4.fit(X_train , y_train)\n",
    "\n",
    "gsearch2_4.grid_scores_, gsearch2_1.best_params_,     gsearch2_1.best_score_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: -0.580120 using {'colsample_bytree': 0.7, 'subsample': 0.8}\n"
     ]
    }
   ],
   "source": [
    "# summarize results\n",
    "print(\"Best: %f using %s\" % (gsearch2_4.best_score_, gsearch2_4.best_params_))\n",
    "# test_means = gsearch2_4.cv_results_[ 'mean_test_score' ]\n",
    "# test_stds = gsearch2_4.cv_results_[ 'std_test_score' ]\n",
    "# train_means = gsearch2_4.cv_results_[ 'mean_train_score' ]\n",
    "# train_stds = gsearch2_4.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "# pd.DataFrame(gsearch2_4.cv_results_).to_csv('my_subsample_colsample_bytree_1.csv')\n",
    "\n",
    "# # plot results\n",
    "# test_scores = np.array(test_means).reshape(len(max_depth), len(min_child_weight))\n",
    "# train_scores = np.array(train_means).reshape(len(max_depth), len(min_child_weight))\n",
    "\n",
    "# for i, value in enumerate(max_depth):\n",
    "#     pyplot.plot(min_child_weight, -test_scores[i], label= 'test_max_depth:'   + str(value))\n",
    "# #for i, value in enumerate(min_child_weight):\n",
    "# #    pyplot.plot(max_depth, train_scores[i], label= 'train_min_child_weight:'   + str(value))\n",
    "    \n",
    "# pyplot.legend()\n",
    "# pyplot.xlabel( 'max_depth' )                                                                                                      \n",
    "# pyplot.ylabel( 'Log Loss' )\n",
    "# pyplot.savefig('my_subsample_colsample_bytree_1.png' )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 又优化了一点，colsample_bytree=0.7，subsample=0.8"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 第五步将正则项调优"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'reg_alpha': [1.5, 2], 'reg_lambda': [0.5, 1, 2]}"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reg_alpha = [ 1.5, 2]    #default = 0, 测试0.1,1，1.5，2\n",
    "reg_lambda = [0.5, 1, 2]      #default = 1，测试0.1， 0.5， 1，2\n",
    "\n",
    "param_test5_1 = dict(reg_alpha=reg_alpha, reg_lambda=reg_lambda)\n",
    "param_test5_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=StratifiedKFold(n_splits=5, random_state=3, shuffle=True),\n",
       "       error_score='raise',\n",
       "       estimator=XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=0.7,\n",
       "       colsample_bytree=0.8, gamma=0, learning_rate=0.1, max_delta_step=0,\n",
       "       max_depth=7, min_child_weight=5, missing=None, n_estimators=200,\n",
       "       n_jobs=1, nthread=None, objective='multi:softprob', random_state=0,\n",
       "       reg_alpha=0, reg_lambda=1, scale_pos_weight=1, seed=3, silent=True,\n",
       "       subsample=0.8),\n",
       "       fit_params=None, iid=True, n_jobs=2,\n",
       "       param_grid={'reg_alpha': [1.5, 2], 'reg_lambda': [0.5, 1, 2]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring='neg_log_loss', verbose=0)"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb2_5 = XGBClassifier(\n",
    "        learning_rate =0.1,\n",
    "        n_estimators=200,  #最优值\n",
    "        max_depth=7,\n",
    "        min_child_weight=5,\n",
    "        gamma=0,\n",
    "        subsample=0.8,\n",
    "        colsample_bytree=0.8,\n",
    "        colsample_bylevel = 0.7,\n",
    "        objective= 'multi:softprob',\n",
    "        seed=3)\n",
    "\n",
    "#采用GridSearchCV来查看最优的树的行列采样\n",
    "gsearch2_5 = GridSearchCV(xgb2_5, param_grid = param_test5_1, scoring='neg_log_loss',n_jobs=2, cv=kfold) \n",
    "gsearch2_5.fit(X_train , y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: -0.580305 using {'reg_alpha': 1.5, 'reg_lambda': 0.5}\n"
     ]
    }
   ],
   "source": [
    "print(\"Best: %f using %s\" % (gsearch2_5.best_score_, gsearch2_5.best_params_))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 得到最佳的参数reg_alpha=1.5，reg_lambda=0.5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 最终我们降低学习率来生成最终模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "y = train[\"interest_level\"]\n",
    "X = train.drop(\"interest_level\",axis=1)\n",
    "#对数据集进行切分\n",
    "X_train, X_test, y_train, y_test = train_test_split(X_train, y_train, test_size=0.3, random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: -0.584013 using {'learning_rate': 0.1}\n"
     ]
    }
   ],
   "source": [
    "\n",
    "parameters= [{'learning_rate':[0.01,0.05,0.1,0.15,0.3]}]\n",
    "xgb6 = XGBClassifier(\n",
    "        learning_rate =0.1,\n",
    "        n_estimators=200,  #最优值\n",
    "        max_depth=7,\n",
    "        min_child_weight=5,\n",
    "        gamma=0,\n",
    "        subsample=0.8,\n",
    "        colsample_bytree=0.7,\n",
    "        colsample_bylevel = 0.7,\n",
    "    \n",
    "        objective= 'multi:softprob',\n",
    "        reg_alpha=1.5,\n",
    "        reg_lambda=0.5,\n",
    "        seed=3)\n",
    "\n",
    "clf = GridSearchCV(xgb6, param_grid=parameters,scoring='neg_log_loss',n_jobs=-1,cv=5)  \n",
    "clf.fit(X_train, y_train)\n",
    "print(\"Best: %f using %s\" % (clf.best_score_, clf.best_params_))\n",
    "##clf.grid_scores_, clf.best_params_, clf.best_score_\n",
    "# print(clf.best_params_)\n",
    "# y_true, y_pred = y_test, clf.predict(X_test)\n",
    "# print(\"Accuracy : %.4g\" % metrics.accuracy_score(y_true, y_pred) )\n",
    "# y_proba=clf.predict_proba(X_test)[:,1]\n",
    "# print(\"AUC Score (Train): %f\" % metrics.roc_auc_score(y_true, y_proba) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 看来最佳学习率就是0.1了 现在所有参数的模型都确定好了，可以生成测试数据了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 1, 2, ..., 2, 1, 2], dtype=int64)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_print = clf.predict(test)\n",
    "y_print"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_print = pd.DataFrame({'InterestLevel':y_print})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bathrooms</th>\n",
       "      <th>bedrooms</th>\n",
       "      <th>price</th>\n",
       "      <th>price_bathrooms</th>\n",
       "      <th>price_bedrooms</th>\n",
       "      <th>room_diff</th>\n",
       "      <th>room_num</th>\n",
       "      <th>Year</th>\n",
       "      <th>Month</th>\n",
       "      <th>Day</th>\n",
       "      <th>...</th>\n",
       "      <th>walk</th>\n",
       "      <th>walls</th>\n",
       "      <th>war</th>\n",
       "      <th>washer</th>\n",
       "      <th>water</th>\n",
       "      <th>wheelchair</th>\n",
       "      <th>wifi</th>\n",
       "      <th>windows</th>\n",
       "      <th>work</th>\n",
       "      <th>InterestLevel</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2950</td>\n",
       "      <td>1475.000000</td>\n",
       "      <td>1475.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>2850</td>\n",
       "      <td>1425.000000</td>\n",
       "      <td>950.000000</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>24</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3758</td>\n",
       "      <td>1879.000000</td>\n",
       "      <td>1879.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>3300</td>\n",
       "      <td>1650.000000</td>\n",
       "      <td>1100.000000</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2.0</td>\n",
       "      <td>2</td>\n",
       "      <td>4900</td>\n",
       "      <td>1633.333333</td>\n",
       "      <td>1633.333333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2016</td>\n",
       "      <td>4</td>\n",
       "      <td>12</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 228 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   bathrooms  bedrooms  price  price_bathrooms  price_bedrooms  room_diff  \\\n",
       "0        1.0         1   2950      1475.000000     1475.000000        0.0   \n",
       "1        1.0         2   2850      1425.000000      950.000000       -1.0   \n",
       "2        1.0         1   3758      1879.000000     1879.000000        0.0   \n",
       "3        1.0         2   3300      1650.000000     1100.000000       -1.0   \n",
       "4        2.0         2   4900      1633.333333     1633.333333        0.0   \n",
       "\n",
       "   room_num  Year  Month  Day      ...        walk  walls  war  washer  water  \\\n",
       "0       2.0  2016      6   11      ...           0      0    0       0      0   \n",
       "1       3.0  2016      6   24      ...           0      0    1       0      0   \n",
       "2       2.0  2016      6    3      ...           0      0    0       0      0   \n",
       "3       3.0  2016      6   11      ...           0      0    0       0      0   \n",
       "4       4.0  2016      4   12      ...           0      0    1       0      0   \n",
       "\n",
       "   wheelchair  wifi  windows  work  InterestLevel  \n",
       "0           0     0        0     0              1  \n",
       "1           0     0        0     0              1  \n",
       "2           0     0        0     0              2  \n",
       "3           1     0        0     0              2  \n",
       "4           0     0        0     0              2  \n",
       "\n",
       "[5 rows x 228 columns]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result =pd.concat([test,y_print],axis=1)\n",
    "result.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "result.to_csv(\"XgBoostWorke.csv\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
